Skip to content

feat: Integrate wheel repairer #1009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

LecrisUT
Copy link
Collaborator

@LecrisUT LecrisUT commented Mar 4, 2025

The goal of this is:

  • Get the linked libraries from the configure step
  • Check if the paths start with the current build environment's sysconfig.get_path("platlib")
    (pip fake venv is really annoying here)
  • If so patch the RPATH to be relative paths
  • Patch existing RPATHs of the current wheel files, e.g. from CMAKE_INSTALL_RPATH_USE_LINK_PATH
  • For windows systems, add dlls to Scripts path.
    Maybe would be better to use the existing paths instead.
  • Do some other magic that delvewheel does
    Afaict, the only special thing it does is inject os.add_dll_directory. Cannot support subprocess.run right now, but we could do it if we generate the wrappers. Will make a subsequent PR for that.

To discuss:

  • Should this be separated as a standalone plugin and instead expose an entry-point scikit-build.repair-wheel?
  • Initially I was considering using auditwheel/delocate/delvewheel directly, but these do not handle differently the packages in site-package vs external
  • Do we extend the interface to pick up system packages as well? I was thinking that the repairwheel trio can do all the hard work generally, but maybe the user needs more nuanced configuration, e.g. with Qt/PySide the requirement is to bundle the Qt framework uniquely, compared to dlevewheel which creates separate dlls for each wheel.

Depends-on: #1022 #1016

@LecrisUT LecrisUT force-pushed the feat/repair-wheel branch 4 times, most recently from 45d04fd to 04d9881 Compare March 10, 2025 12:05
@LecrisUT

This comment was marked as resolved.

@LecrisUT LecrisUT force-pushed the feat/repair-wheel branch 25 times, most recently from 81f9407 to 9cc8ea9 Compare March 17, 2025 14:13
@FabianSchuetze
Copy link

Thanks for this PR. Should it already be working? I tried commits 9700ce0 and b4652c5 and got the following error (with b4652c5):

  [notice] A new release of pip is available: 24.0 -> 25.1.1
  [notice] To update, run: pip install --upgrade pip
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  Traceback (most recent call last):
    File "/tmp/quant-vit-main/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/tmp/quant-vit-main/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/quant-vit-main/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
      return hook(metadata_directory, config_settings)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/build/__init__.py", line 94, in prepare_metadata_for_build_wheel
      from .wheel import _build_wheel_impl
    File "/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/build/wheel.py", line 23, in <module>
      from ..repair_wheel import WheelRepairer
    File "/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/repair_wheel/__init__.py", line 19, in <module>
      from .darwin import MacOSWheelRepairer
    File "/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/repair_wheel/darwin.py", line 9, in <module>
      from .rpath import RpathWheelRepairer
    File "/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/repair_wheel/rpath.py", line 13, in <module>
      from . import WheelRepairer
  ImportError: cannot import name 'WheelRepairer' from partially initialized module 'scikit_build_core.repair_wheel' (most likely due to a circular import) (/tmp/pip-build-env-beb56syl/overlay/lib/python3.11/site-packages/scikit_build_core/repair_wheel/__init__.py)
  error: subprocess-exited-with-error

I set up my pjproject.toml as:

[build-system]
requires = [
  "scikit-build-core @ git+https://github.com/scikit-build/scikit-build-core.git@b4652c56849ad1a0d23246b17cd37ff558c30a93"
]
build-backend = "scikit_build_core.build"

@LecrisUT
Copy link
Collaborator Author

Thanks for this PR. Should it already be working?

It used to work, but I think I did a hasty refactor and broke it. I'll try to get it back to working state this week. There were some unresolved failures, but I don't think there were on the linux builders. There is an issue with patchelf on Fedora distros (and probably many other), but I had a version using lief which I will try to resurrect now that there have been some improvements upstream.

@LecrisUT LecrisUT force-pushed the feat/repair-wheel branch 2 times, most recently from 893377e to 928e504 Compare July 30, 2025 16:45
LecrisUT added 15 commits July 30, 2025 18:54
Signed-off-by: Cristian Le <[email protected]>
Signed-off-by: Cristian Le <[email protected]>
`auditwheel` pacher forces `DT_RPATH`. Here instead we use `DT_RUNPATH` so that it can be overwritten by the user

Signed-off-by: Cristian Le <[email protected]>
Signed-off-by: Cristian Le <[email protected]>
This is needed because the last delocate package that supports python 3.8 does not have _get_rpaths

Signed-off-by: Cristian Le <[email protected]>
Signed-off-by: Cristian Le <[email protected]>
@LecrisUT LecrisUT force-pushed the feat/repair-wheel branch from 928e504 to ce63083 Compare July 30, 2025 17:25
@LecrisUT
Copy link
Collaborator Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants