NPM + Docker publishing off main is currently broken: "too many versions"
#38341
Replies: 4 comments 7 replies
-
|
Digging into this via the not widely documented NPM registry APIs and some help from #!/usr/bin/env bash
set -euo pipefail
PACKAGE="renovate" # <-- your package name
KEEP_COUNT=5 # keep the latest 5 releases (excluding tags)
# --------------------------------------------------------------------
# 1️⃣ Grab full package metadata (versions + publish times)
# --------------------------------------------------------------------
npm view "$PACKAGE" time --json | jq -r '
del(.created,.modified) |
to_entries | map({v: .key, t: .value}) |
sort_by(.t)
' > all-versions.json
# --------------------------------------------------------------------
# 2️⃣ Pull the *downloads per version* map for the last week
# --------------------------------------------------------------------
DL_JSON=$(curl -s https://api.npmjs.org/versions/${PACKAGE}/last-week)
# Convert the JSON object into a Bash associative array
declare -A DL
while IFS=$'\t' read -r ver cnt; do
DL["$ver"]=$cnt
done < <(echo "$DL_JSON" | jq -r '.downloads | to_entries[] | "\(.key)\t\(.value)"')
# Helper that answers “downloads <= 300” for a given version
low_downloads() {
local ver=$1
local dl=${DL["$ver"]:-0} # 0 if the key is missing
(( dl <= 300 ))
}
# TODO add back
# TODO add back
# --------------------------------------------------------------------
# 3️⃣ Check that nobody depends on the package
# --------------------------------------------------------------------
# DEPENDS=$(curl -s "https://registry.npmjs.org/-/package/${PACKAGE}/metadata?fields=dependents" | jq -r '.dependents')
#
# if (( DEPENDS != 0 )); then
# echo "❌ ${PACKAGE} has ${DEPENDS} dependents – you cannot unpublish any version."
# exit 1
# fi
# TODO add back
# TODO add back
# --------------------------------------------------------------------
# 4️⃣ Build the list of candidates (old + low‑downloads)
# --------------------------------------------------------------------
NOW=$(date +%s) # epoch seconds
OLD_VERSIONS=()
while read -r ver ts; do
pub=$(date -d "$ts" +%s)
age=$(( NOW - pub ))
(( age > 259200 )) && OLD_VERSIONS+=("$ver") # > 72h
done < <(jq -r '.[] | [.v,.t] | @tsv' all-versions.json)
CANDIDATES=()
for ver in "${OLD_VERSIONS[@]}"; do
low_downloads "$ver" && CANDIDATES+=("$ver")
done
echo "❗ ${#CANDIDATES[@]} versions meet the <300 downloads & 72 h criteria."
# --------------------------------------------------------------------
# 5️⃣ Preserve the newest releases (plus any dist‑tags)
# --------------------------------------------------------------------
# a) keep the latest N releases (regardless of tags)
ALL_VERS=( $(jq -r '.[] | .v' all-versions.json) ) # already sorted old→new
KEEP_RELEASES=( "${ALL_VERS[@]: -$KEEP_COUNT}" ) # last N entries
# b) keep anything that has a dist‑tag (latest, beta, …)
TAGS=$(npm view "$PACKAGE" dist-tags --json | jq -r 'keys[]')
for tag in $TAGS; do
tag_ver=$(npm view "$PACKAGE@${tag}" version)
KEEP_RELEASES+=( "$tag_ver" )
done
KEEP_RELEASES=( $(printf "%s\n" "${KEEP_RELEASES[@]}" | sort -u) )
# --------------------------------------------------------------------
# 6️⃣ Final delete list – candidates minus keep list
# --------------------------------------------------------------------
TO_UNPUBLISH=()
for ver in "${CANDIDATES[@]}"; do
if ! printf '%s\n' "${KEEP_RELEASES[@]}" | grep -qFx "$ver"; then
TO_UNPUBLISH+=("$ver")
fi
done
echo "✔️ Will delete ${#TO_UNPUBLISH[@]} safe versions:"
printf ' %s\n' "${TO_UNPUBLISH[@]}"
# --------------------------------------------------------------------
# 7️⃣ Unpublish (commented out – run after a dry‑run)
# --------------------------------------------------------------------
# for ver in "${TO_UNPUBLISH[@]}"; do
# echo "Unpublishing ${PACKAGE}@${ver} ..."
# npm unpublish "${PACKAGE}@${ver}" --force
# done |
Beta Was this translation helpful? Give feedback.
5 replies
-
|
This unfortunately "hallicunates" the |
Beta Was this translation helpful? Give feedback.
-
|
Running this, we can see the following list of Details |
Beta Was this translation helpful? Give feedback.
-
Via script: Details# Co-authored-by: gpt-oss:20b
versions = [
# ...
]
counts = {}
for v in versions:
major = v.split('.')[0] # take the part before the first dot
counts[major] = counts.get(major, 0) + 1
# Build and print the Markdown table
print("| Major version | Count |")
print("|---------------|-------|")
for major in sorted(counts, key=int):
print(f"| {major} | {counts[major]} |") |
Beta Was this translation helpful? Give feedback.
-
|
We should keep at least 1 (one) version from each major release A as starting point, I only want to modify the 1.x releases, to make sure we're able to touch things that should very much not be used any more |
Beta Was this translation helpful? Give feedback.
-
|
Currently testing: However: |
Beta Was this translation helpful? Give feedback.
-
|
From NPM support:
We'll shortly be providing the list |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
We're looking at removing these ~3000 NPM tags: This would then keep the following last-release-of-each-major: |
Beta Was this translation helpful? Give feedback.
-
|
sounds good 👍 |
Beta Was this translation helpful? Give feedback.
-
|
NPM support are currently (manually) unpublishing the versions noted here ☝️ I'll update once I've heard back from them |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Thanks to support from NPM support, releases are working again 🚀 https://www.npmjs.com/package/renovate/v/41.135.3 / https://github.com/renovatebot/renovate/releases/tag/41.135.3 is the first successful release |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Separate to yesterday's incident #38322, we are also seeing issues currently with publishes to NPM.
While the maintainers had started investigating the logs, wonderful contributor @astellingwerf managed to find the root cause (#38338):
We're currently working with NPM support, and are separately investigating versions that we can safely unpublish, as per the npm registry guidelines.
As noted in #38353 this also impacts the Docker image builds
Beta Was this translation helpful? Give feedback.
All reactions