Default prune-cache: true causes unnecessary PyPI downloads for pre-built wheels

[ei-grad]

Summary

The default prune-cache: true removes pre-built wheels from cache, causing them to be re-downloaded from PyPI on every CI run. This generates unnecessary load on PyPI infrastructure.

What gets pruned vs retained

Cache content	Pruned?	Effect
sdists	❌ Retained	No re-download
Locally-built wheels	❌ Retained	No rebuild
Pre-built wheels	✅ Pruned	Re-downloaded every run

Most popular packages (numpy, pandas, requests, etc.) distribute pre-built wheels and are affected.

Evidence

Reproduction: https://github.com/ei-grad/uv-cache-issue

Pre-built wheels re-downloaded despite cache hit

Second run, prune-cache: true — job log:

Cache hit for: setup-uv-1-...-with-prune
Cache Size: ~434 MB

Downloading numpy (15.8MiB)
Downloading pandas (10.4MiB)

Locally-built wheels served from cache

First run — run 21582159578:

Building pyspark==4.1.1
Built pyspark==4.1.1

Second run — job log:

 + pyspark==4.1.1

No rebuild — locally-built wheel served from cache.

With prune-cache: false — no PyPI downloads

Job log:

Cache hit for: setup-uv-1-...-without-prune
Cache Size: ~1758 MB

Installed 15 packages in 74ms

Impact

Given setup-uv's popularity, the default causes significant unnecessary traffic to PyPI across thousands of CI pipelines.

Suggestion

Not sure I have a good solution. Maybe it would help to make prune-cache keep wheels?

[eifinger]

Hi. It looks like this issue does not add any new information over #54 (comment). Am I missing something?

[ei-grad]

This issue is quite different - it is not about enabling the cache, but about setup-uv action behavior and absence of "lets don't download anything from PyPI in every CI job" option. And this issue adds verified information on current behavior which is probably unexpected for people who want to enable caching.

[edit, 9 Feb] to clarify: option to "don't download" exists, "prune-cache: false", but it wouldn't find it's way to users config since it makes CI cache restoring step to be longer than the job without cache in some (many?) scenarios - I believe the huge difference in size, demonstrated in the issue description, is the real problem

[eifinger]

Isn't this issue about the exact same thing as comment #54 (comment)?

I am very much in favor of easing the load on a service PyPi/fastly are offering for free, I do however still see the impact I outlined in the comment I linked above. There is a risk of unforeseen costs for users of this action as I see it. Happy to be proven wrong though.

[ei-grad]

Isn't this issue about the exact same thing as comment #54 (comment)?

Correct.

[ei-grad]

Current enable-cache: auto behavior looks good to me. As a user, I care about both PyPI load and efficient .venv recreation in CI. The pruning logic is close to optimal - it just needs to also keep pre-built wheels, not only sdists and locally-built wheels.

Re: unforeseen costs - I think keeping pre-built wheels wouldn't dramatically increase cache size. I thought most of the 1.7GB -> 435MB reduction comes from pruning other artifacts, not wheels. Will measure/recheck if it is true/enough though.

[eifinger]

We have raised this issue with the PyPI admins to get their thoughts on this.

[miketheman]

I'm one of the PyPI Admins, but I'm not speaking on behalf of all of us - just my opinions here.

It seems that caching things already downloaded once and unlikely to change (thanks to lockfiles, hashes, etc) is in the best interest to all involved - nothing groundbreaking with this idea, right?

From reading this thread, it seems like having a default that tells setup-uv to not cache prebuilt wheels, uv will then download all the same exact wheels each time from PyPI. Is that a worthwhile use of everyone's bandwidth? It's already been done, so save it instead of fetching from upstream again. That's what I understand one of the reasons uv is fast on local installations - there's a local global cache of things it's already downloaded.

From an impact estimate, uv's attraction of speed is non-trivial, and has worked its way into many CI installations. I did some querying of the BigQuery dataset, and came up with some conclusions. I'd invite you to check my results, and perform your own calculations in case I'm wrong!

Source data, query and charts

Looking at 7 days of download data where installers report CI=true (see pypa/pip#5499 for background on this flag) shows:

Pretty impressive - uv is responsible for roughly 50% of all downloads from PyPI in environments where the installer reports CI=true. I picked 7 days since that's already a lot of data for BQ to crunch on, if you have large budgets, credits, feel free to query for longer timeframes).

Focusing a query on a single package boto3 which for a longer time period (I don't always understand BQ's data estimates, but hey...) shows an even more revealing picture:

The most popular package gets downloaded a lot more frequently by uv in CI environments than anywhere else - responsible for ~65-75% of all downloads from PyPI.

Does this mean that other installers are using caching "more correctly", or are doing things less often, or simply not advertising it? It's hard to say conclusively, but it's clear that uv is being used more than before and is downloading often from PyPI.

I found this line interesting in the uv docs:

With uv, it turns out that it's often faster to omit pre-built wheels from the cache (and instead re-download them from the registry on each run).

If PyPI was a paid-for-by-users service, there's a clear trade off speed vs cost here - cache (and be potentially slower) and pay less for the service, or don't cache pay more and get speed.
There's no cost the user has to pay to PyPI for downloads, so PyPI (and the donations from Fastly) are effectively incentivizing further consumption, which leads to higher consumption.

I suspect that setting the default behavior to cache what has already been downloaded will be fine for 90% of the use cases out there, and having a toggle to switch over for those most latency dependent and willing to push more load onto PyPI should be available. The documentation should express what the "cost" of uncached is - namely raising costs of running PyPI (yes, even if they are donated).

(I'll also hazard that most users think they are doing the Right Thing when using the provided action's defaults - the prune behavior was also surprising to me!)

Anyhow, that's a lot of words and pictures. Hope this makes sense, let me know if it doesn't!

P.S. a talk by some folk you probably know at PyCon US 2024 may prove interesting, as well as this post