ci: add a upgrade bats test #3621
Conversation
|
Need this to make sure we don't break stuff after our cache refactoring. |
There was a problem hiding this comment.
Pull request overview
This PR adds upgrade testing capabilities for the zot container registry by introducing two new BATS test files that verify compatibility when upgrading from a released version to a development version. The tests download the latest release binary, run comprehensive tests against it, then upgrade to the new binary and repeat the tests to ensure backward compatibility.
- Adds upgrade test infrastructure with helper functions to download and run release binaries
- Creates comprehensive upgrade test suites for both full and minimal zot builds
- Integrates new tests into the CI pipeline
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| test/ports.json | Allocates port ranges (11500-11509 and 11510-11519) for the two new upgrade test suites |
| test/blackbox/upgrade.bats | Comprehensive upgrade test suite for full zot build, testing release→upgrade→new binary flow |
| test/blackbox/upgrade_minimal.bats | Comprehensive upgrade test suite for minimal zot build, testing release→upgrade→new binary flow |
| test/blackbox/helpers_zot.bash | Adds helper functions zot_rel_serve and zot_rel_min_serve to download and run release binaries |
| test/blackbox/ci.sh | Registers the new upgrade test suites in the CI test array |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rchincha
force-pushed
the
upgrade-ci
branch
2 times, most recently
from
3108964 to
61cc513
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3621 +/- ##
=======================================
Coverage 91.53% 91.53%
=======================================
Files 186 186
Lines 26542 26542
=======================================
Hits 24295 24295
Misses 1454 1454
Partials 793 793 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
rchincha
force-pushed
the
upgrade-ci
branch
3 times, most recently
from
b6e4af6 to
53565dd
Compare
rchincha
force-pushed
the
upgrade-ci
branch
2 times, most recently
from
aba89ae to
8ba283d
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 17 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
rchincha
force-pushed
the
upgrade-ci
branch
4 times, most recently
from
8087d08 to
328631f
Compare
rchincha
force-pushed
the
upgrade-ci
branch
3 times, most recently
from
a913ae9 to
292780e
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
andaaron
left a comment
andaaron
left a comment
There was a problem hiding this comment.
How long do these tests take? Does it make sense to run them as a separate make target / GH workflow?
| checksum_url="https://github.com/project-zot/zot/releases/latest/download/checksums.sha256.txt" | ||
| checksum_file="${BATS_FILE_TMPDIR}/zot-sha256sums.txt" | ||
| curl -L -o "${checksum_file}" "${checksum_url}" | ||
| expected_sum=$(grep "zot-${OS}-${ARCH}$" "${checksum_file}" | awk '{print $1}') |
There was a problem hiding this comment.
Nitpick: maybe this could be simplified by using something like "$(cat ${checksum_file}) ${zot_path}" | sha256sum --check --status
Same comment for below.
|
|
||
| @test "[new] push image" { | ||
| zot_port=`cat ${BATS_FILE_TMPDIR}/zot.port` | ||
| run skopeo --insecure-policy copy --dest-tls-verify=false \ |
There was a problem hiding this comment.
While overwriting make sense as a test case. I think doing it first interferes with any validation we want to do on verifying the previously pushed image to the older zot version is still accessible.
There was a problem hiding this comment.
We can check existence first and then push again. We are already pushing a entirely new image.
Fixes project-zot#3601 Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
rchincha
force-pushed
the
upgrade-ci
branch
2 times, most recently
from
b13d88b to
b56455f
Compare
Signed-off-by: Ramkumar Chinchani <[email protected]>
Signed-off-by: Ramkumar Chinchani <[email protected]>
2 participants
Fixes #3601
What type of PR is this?
Which issue does this PR fix:
What does this PR do / Why do we need it:
If an issue # is not available please add repro steps and logs showing the issue:
Testing done on this change:
Automation added to e2e:
Will this break upgrades or downgrades?
Does this PR introduce any user-facing change?:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.