Skip to content

Add support for extra variants of relative urls for simple html pypi API - #174

Merged
ryanking13 merged 4 commits into
pyodide:mainfrom
artem-samokhin:better-relative-url-check
Jan 21, 2025
Merged

Add support for extra variants of relative urls for simple html pypi API#174
ryanking13 merged 4 commits into
pyodide:mainfrom
artem-samokhin:better-relative-url-check

Conversation

@artem-samokhin

@artem-samokhin artem-samokhin commented Jan 20, 2025

Copy link
Copy Markdown
Contributor

Why:
I have a private python packages repository powered by Sonatype Nexus Repository software.
This repo unfortunatelly only supports simple HTTP API, and, additionally, list of packages from Nexus looks like that:
GET https://self-hosted-nexus-repo.example/repository/pypi/simple/example-python-package/

<!DOCTYPE html>
<html lang="en">
<head><title>Links for example-python-package</title>
  <meta name="api-version" value="2"/>
</head>
<body><h1>Links for example-python-package</h1>
        <a href="../../packages/example-python-package/0.0/example_python_package-0.0.tar.gz#sha256=<hash>" rel="internal" data-requires-python="&gt;=2.7,!=3.0,!=3.1,!=3.2,!=3.3" >example_python_package-0.0.tar.gz</a><br/>
        <a href="../../packages/example-python-package/0.1/example_python_package-0.1-py2.py3-none-any.whl#sha256=<hash>" rel="internal" data-requires-python="&gt;=3.4" >example_python_package-0.1-py2.py3-none-any.whl</a><br/>
        ...
</body>
</html>

And because urls starts with ., micropip thinks that they are not relative urls and does not append index url to it.
So, for example, if we are installing example-python-package==0.1 from web page called my-web-page.example, web brouser receives GET request to ../../packages/example-python-package/0.1/example_python_package-0.1-py2.py3-none-any.whl#sha256=<hash> from micropip and interpretes it as https://my-web-page.example/repository/pypi/packages/example-python-package/0.1/example_python_package-0.1-py2.py3-none-any.whl#sha256=<hash> instead of https://self-hosted-nexus-repo.example/packages/example-python-package/0.1/example_python_package-0.1-py2.py3-none-any.whl#sha256=<hash>.

What: better check for relative vs absolute url when dealing with simple HTTP API.

P.S. I can't figure out how to make tests that require spawn_web_server to work properly. But my part should not affect those, so I hope it is fine.

@ryanking13 ryanking13 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks. Could you please update the changelog?

<html>
<head>
<meta name="pypi:repository-version" content="1.1">
<title>Links for fake-pkg-micropip-test</title>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please update the filename? So the readers can easily know that it is for testing the relative url.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, thanks, I've updated changelog (I assume I have to put my changes under Unreleased section) and renamed new fixtures file.

@ryanking13
ryanking13 merged commit 5ce6e1f into pyodide:main Jan 21, 2025
@hoodmane

Copy link
Copy Markdown
Member

Thanks @artem-samokhin!

@agriyakhetarpal agriyakhetarpal mentioned this pull request Jan 29, 2025
@agriyakhetarpal agriyakhetarpal added this to the 0.9.0 milestone Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants