[Bug 2075477] Re: SRU: Backport LLVM 18.1.8 to Ubuntu LTS

[Zixing Liu]

** Description changed:

  SRU: Backport LLVM 18.1.8 to Ubuntu 24.04 LTS
  
- A test rebuild using these packages was done as described in LP:
+ [ Impact ]
+ 
+  * A newer Clang/LLVM version will be available for older releases as a
+ non-default LLVM selection.
+ 
+  * On newer releases which had an older LLVM 18 release: some general
+ bug fixes and improvements that release note may or may not mention:
+ https://discourse.llvm.org/t/18-1-8-released/79725 (and mostly about
+ PowerPC fixes in 18.1.7).
+ 
+ [ Test Plan ]
+ 
+  * Run autopkgtests on reverse dependencies.
+ 
+  * On LTS releases which don't currently have LLVM 18, we can just
+ examine the builtin unittest results, the unit tests don't need to have
+ a 100% pass rate because that's impossible even for the upstream
+ sometimes.
+ 
+ [ Where problems could occur ]
+ 
+  * LLVM is a very large piece of software with a lot of moving parts. A
+ lot could go wrong even in a minor release: optimizer misoptimizations,
+ Clang behavior changes, library linkage changes, C/C++ UB (undefined
+ behavior) handling changes, on-disk IR format change (maybe
+ compatible/upgradable), change of "internal" C++ API parameters, and
+ change of C API behaviors.
+ 
+  * In the backport, since we will need to apply extra patches for LLVM
+ to compile in an older LTS, more LLVM breakages may happen unnoticed as
+ we don't really care about unittests provided by LLVM (because a lot of
+ them don't function correctly on non-amd64 and non-arm64 platforms; a
+ lot of LIT tests may require the build system to have the same
+ environment as the upstream CI, which is by itself a moving target.).
+ 
+  * In Focal backport, a RISC-V patch was added to fix LLVM code
+ generation and internal linker behavior when targeting RISC-V. This is
+ needed because LLVM is by default a cross-compiler that could generate
+ any supported machine code on any supported platform (we also enabled
+ LLVM Orc JIT remoting functionality, which allows LLVM JIT to transform
+ LLVM IR into machine code over the network). This may pose extra risks
+ since our binutils on Focal only supports RISC-V ABI v2.0 while LLVM 18
+ speaks RISC-V ABI v2.1 (whose linker identity string is incompatible
+ with v2.0, which is quite counter-intuitive). The patch is needed for
+ older binutils to correctly link the RISC-V binaries when the RISC-V
+ targets contains these RISC-V standard CPU extensions: +I (i2p0) +M
+ (i2p0) +A (i2p0) +F (i2p0) +D (i2p0) +Zicsr (i2p0 when using +G with
+ binutils in 20.04, otherwise not understood with i2p1) +Zifencei
+ +Zipause
+ 
+ [ Other Info ]
+ 
+  * A test rebuild using these packages was done as described in LP:
  #2076024.
  
- The llvm-18 testsuite run during the build doesn't show regressions.
+  * The llvm-18 testsuite run during the build doesn't show regressions.
  
- The update exposes an underlinking of bpftrace, which should be SRUed at
- the same time. See LP: #2081848
+  * The update exposes an underlinking of bpftrace, which should be SRUed
+ at the same time. See LP: #2081848

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2075477

Title:
  SRU: Backport LLVM 18.1.8 to Ubuntu LTS

Status in llvm-toolchain-18 package in Ubuntu:
  Confirmed
Status in llvm-toolchain-18 source package in Focal:
  Confirmed
Status in llvm-toolchain-18 source package in Jammy:
  Confirmed
Status in llvm-toolchain-18 source package in Noble:
  Fix Committed

Bug description:
  SRU: Backport LLVM 18.1.8 to Ubuntu 24.04 LTS

  [ Impact ]

   * A newer Clang/LLVM version will be available for older releases as
  a non-default LLVM selection.

   * On newer releases which had an older LLVM 18 release: some general
  bug fixes and improvements that release note may or may not mention:
  https://discourse.llvm.org/t/18-1-8-released/79725 (and mostly about
  PowerPC fixes in 18.1.7).

  [ Test Plan ]

   * Run autopkgtests on reverse dependencies.

   * On LTS releases which don't currently have LLVM 18, we can just
  examine the builtin unittest results, the unit tests don't need to
  have a 100% pass rate because that's impossible even for the upstream
  sometimes.

  [ Where problems could occur ]

   * LLVM is a very large piece of software with a lot of moving parts.
  A lot could go wrong even in a minor release: optimizer
  misoptimizations, Clang behavior changes, library linkage changes,
  C/C++ UB (undefined behavior) handling changes, on-disk IR format
  change (maybe compatible/upgradable), change of "internal" C++ API
  parameters, and change of C API behaviors.

   * In the backport, since we will need to apply extra patches for LLVM
  to compile in an older LTS, more LLVM breakages may happen unnoticed
  as we don't really care about unittests provided by LLVM (because a
  lot of them don't function correctly on non-amd64 and non-arm64
  platforms; a lot of LIT tests may require the build system to have the
  same environment as the upstream CI, which is by itself a moving
  target.).

   * In Focal backport, a RISC-V patch was added to fix LLVM code
  generation and internal linker behavior when targeting RISC-V. This is
  needed because LLVM is by default a cross-compiler that could generate
  any supported machine code on any supported platform (we also enabled
  LLVM Orc JIT remoting functionality, which allows LLVM JIT to
  transform LLVM IR into machine code over the network). This may pose
  extra risks since our binutils on Focal only supports RISC-V ABI v2.0
  while LLVM 18 speaks RISC-V ABI v2.1 (whose linker identity string is
  incompatible with v2.0, which is quite counter-intuitive). The patch
  is needed for older binutils to correctly link the RISC-V binaries
  when the RISC-V targets contains these RISC-V standard CPU extensions:
  +I (i2p0) +M (i2p0) +A (i2p0) +F (i2p0) +D (i2p0) +Zicsr (i2p0 when
  using +G with binutils in 20.04, otherwise not understood with i2p1)
  +Zifencei +Zipause

  [ Other Info ]

   * A test rebuild using these packages was done as described in LP:
  #2076024.

   * The llvm-18 testsuite run during the build doesn't show
  regressions.

   * The update exposes an underlinking of bpftrace, which should be
  SRUed at the same time. See LP: #2081848

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-18/+bug/2075477/+subscriptions