Vendor pypa/packaging into micropip - #178
Conversation
|
Thanks for working on this @agriyakhetarpal! Please ping me when it is ready to be reviewed. |
agriyakhetarpal
changed the title
pypa/packaging for micropippypa/packaging into micropip
| [[tool.mypy.overrides]] | ||
| module = "micropip._vendored.*" | ||
| warn_unreachable = false |
There was a problem hiding this comment.
I've had to keep this as there is some unreachable code in src/packaging/, which is not of concern for us, and TIL that the warn_unreachable option is still raised for excluded files.
Thanks, @ryanking13! I think it is ready now. |
| from typing import Any, Dict, List, Optional, Union, Literal, TypeAlias, TypedDict | ||
|
|
||
| import packaging.utils | ||
| import micropip._vendored.packaging.utils as packaging_utils |
There was a problem hiding this comment.
(No need to address in this PR)
both _vendored and externals directory vendors external packages, so I think we can merge them into a single directory
There was a problem hiding this comment.
Thanks! Yes, I noticed that. I'll open a follow-up PR after this to merge them. I have no preference on the name of the folder, but it should be _externals or _vendored, so that it doesn't form a part of the public API.
There was a problem hiding this comment.
Thanks, I think _vendored is a better name. At the beginning, the external directory contained subset of pip, and it had a _vendor directory in it (ref).
There was a problem hiding this comment.
As packaging is a part of the private API for micropip, we don't need to graft it. However, there is no nice way to exclude specific files without explicitly naming them. This is because MANIFEST.in is processed from top to bottom, like an interpreted language, so we can't exclude the entire first and include only what we want.
The better way would be for us to switch to hatchling in a follow-up item, which supports this kind of use case much better.
There was a problem hiding this comment.
Thanks! Yes, I agree that in terms of managing file inclusion/exclusion, other build backends like hatchling are better than setuptools. It can be a follow up item.
|
Ready for another look! |
There was a problem hiding this comment.
Thanks @agriyakhetarpal! Could you please update the changelog and make a release?
There was a problem hiding this comment.
Thanks! Yes, I agree that in terms of managing file inclusion/exclusion, other build backends like hatchling are better than setuptools. It can be a follow up item.
| from typing import Any, Dict, List, Optional, Union, Literal, TypeAlias, TypedDict | ||
|
|
||
| import packaging.utils | ||
| import micropip._vendored.packaging.utils as packaging_utils |
There was a problem hiding this comment.
Thanks, I think _vendored is a better name. At the beginning, the external directory contained subset of pip, and it had a _vendor directory in it (ref).
|
Thanks for the review, Gyeongjae! I'll merge this and move the |
2 participants
Description
Closes #176, and stemmed off from pyodide/pyodide#5366.
This PR vendors the https://github.com/pypa/packaging project into
micropip, as the issue above noted that it is difficult to install a newer version ofpackaginginto the environment, asmicropipdepends on it and installs it. This meanspackagingcan now be installed separately frommicropip.Changes
from packaging import <...>imports intomicropip._vendored.packaging.src.packaging import <...>packagingfiles are present in both the source distribution and the wheel, excluding non-package-related files, and including the relevant license files.