free threading (no-gil) support

[esc]

Version 0.46.0b1 does not appear to support free-threading.

ref #1281 (comment)

[ngoldbaum]

I tried setting up free-threaded wheels on my fork of llvmlite just now: ngoldbaum#1

Currently the Mac builds fail because 3.14t isn't a valid python version to use when creating a conda environment: https://github.com/ngoldbaum/llvmlite/actions/runs/18954231888/job/54126755450?pr=1#step:3:376

I don't know offhand how to work around that - I think 3.14t builds aren't yet set up for anaconda? Presumably you all know better than I would...

The Linux builds fail trying to install LIEF: https://github.com/ngoldbaum/llvmlite/actions/runs/18954231907/job/54126990672?pr=1#step:4:882.

So I think getting 3.14t LIEF wheels uploaded is probably the first step. See lief-project/LIEF#1255.

[esc]

I don't know offhand how to work around that - I think 3.14t builds aren't yet set up for anaconda? Presumably you all know better than I would...

From what I can tell:

 conda create -n test python=3.14 python-freethreading

should allow you to install an interpreter. However, there are no packages (e.g. PyLIEF) and they are unlikely to become available anytime soon. You could install a no-gil PyLIEF wheel if there was one however. That's the theory and it's a stack of sub-optimality for the time being.

Perhaps -- if you would like to investigate this further -- it may be looking at what cond-forge have to offer on this front? I'd certainly be interested in that..

[esc]

Oh, also, we do have a full-stack building framework here:

https://github.com/numba/numba-hatchery

this could probably be adapted with minimal effort to try and run using a python.org based docker container with a free-threading python on a mac (if such an image were to be available.. that is..)

[ngoldbaum]

I think it should be possible to build PyLIEF from source without going through the pain of properly packaging it. I'm a little concerned about the nanobind leak warnings I see when I build and run PyLIEF locally on 3.14t and wanted to understand what's happening there but haven't had a chance to dive into that. lief-project/LIEF#1255 (comment)

That said, the PyLIEF tests all pass so functionally it should be fine to build llvmlite.

[ngoldbaum]

I was able to get the mac builds to work by building PyLIEF from source, see https://github.com/ngoldbaum/llvmlite/actions/runs/18986468571?pr=1 and https://github.com/ngoldbaum/llvmlite/pull/1/files for the corresponding diff. It's not very fast: building PyLIEF is slow and I need to do it once for the build and another time for the tests. I could probably speed it up by building it once and caching the wheel.

I think a similar change needs to happen in the other wheel-building configurations (e.g. buildscripts/manylinux/build_llvmlite.sh for the manylinux builds). Unfortunately I'm running out of steam today, but hopefully the rest is relatively straightforward.

[rgommers]

it may be looking at what cond-forge have to offer on this front?

The migrator for 3.14t is supposed to be kicking in right about now, but it hasn't started yet: https://conda-forge.org/status/migration/?name=python314t. There are some manually built packages (numpy, scipy et al), but not many. Cc @zklaus who may be able to say more about the migration.

[ngoldbaum]

See #1351 for an attempt at setting up cp314t wheel builds.

[zklaus]

Just to confirm, the conda-forge migration will start as soon as conda-forge/conda-forge-pinning-feedstock#7777 gets merged, which should be any day now.

[stuartarchibald]

Thanks for working on this @ngoldbaum. Having looked at this previously to get Numba working with free-threading... given the Python code is Python version invariant, and the C++ code does not use the Python C-API (it's just a DSO loaded through ctypes), I'm fairly sure there would be no material difference in the executable part of a free-threaded and a non-free-threaded package. This will potentially make building easier as it's potentially "just" a package naming issue. Hope this helps.