Continuous Discussion: LLVM update plan

[jakirkham]

Feature request

Currently it looks like llvmlite is pinned to LLVM 14

There was a question in the past about updating to LLVM 15 ( #914 ). Though, at the time, the pin had just recently been bumped to LLVM 14

More recently it appears there has been some work done on LLVM 15:

• Support building against llvm15 #1035
• LLVM 15 conda recipe #1036
• Test #986 on CI with LLVM 15 #1038

Ofc in the interim LLVM has made a few more releases of its own:

• 16.0.6
• 17.0.6
• 18.1.4

So am curious to hear what the Numba team thinks about updating LLVM. Also interested to know what version(s) is/are being considered for llvmlite pins

Recognize LLVM is a pretty complex library and each update involves a lot of careful testing and thought, which I'm deeply appreciative of

Thanks in advance for your feedback! 🙏

[esc]

FWIW: llvmlite 0.43 will ship with support for building against 15: #1047

[esc]

Also, there is work being done towards 17: #1042 (comment)

[gmarkall]

To answer the question generally - we update to newer versions of LLVM as fast as we are able, but given resource constraints that's not particularly fast.

In addition to the items in the issue description, other work is needed to move forward beyond 15 (no timelines specified, but it would be nice to complete these prior to the 0.44 release):

• Add basic infra required to move Numba to NewPassManager #1046 - adding support for the new pass manager. Numba will also need to be updated to use the new pass manager
• Port refprune pass to NewPassManager #1057 - the refprune pass needs porting to the new pass manager
• Support for opaque pointers, and corresponding Numba changes

Following the successful completion of those items we can then move forward beyond 15.

[BwL1289]

Also interested. I know we're a bit far off, but we're using LLVM 20. Not sure if llvmlite is a sponsored project, but there may be an opportunity to partner with an org to help support it.

[esc]

@jakirkham can this be closed?

[h-vetinari]

can this be closed?

It's kind of an ongoing concern though... If you close this issue, you'll get another one about "when LLVM 16,17,18,19,...". IMO it's more beneficial to keep all the context (like this comment) in one place.

W.r.t. that list, it seems that most of the conditions for updating beyond LLVM 15 are now met? (support for opaque pointers was merged in #1064 AFAICT, though #900 is still open 🤔).

[BwL1289]

Agreed with @h-vetinari. Also linking 1092

[esc]

@h-vetinari @BwL1289 ok, thank you for your feedback I have updated the title, added a label and pinned this issue to reflect that it has such a status. On going discussion for current and future LLVM support.

[jack-mcivor]

Generally speaking, what is the benefit to numba users for upgrading the bundled LLVM? Do these usually come with performance improvements for compiling user code & run time of user code?

[gmarkall]

Generally speaking, what is the benefit to numba users for upgrading the bundled LLVM? Do these usually come with performance improvements for compiling user code & run time of user code?

That is generally intended to be the case. For specific recent examples, @yashssh may be able to give particular details for the CPU target. In general it's a good idea to keep up with current LLVM because older versions are not maintained, and there are APIs that we want to move to in new versions, such as the new pass manager (which we recently moved to), and OrcJIT / JITLink.