chore: Release v1.9.0

[arya2]

Part of #8543.
Will close #8709.

This PR still needs to set an NU6 activation height for Testnet and to update the NU6 funding streams start height. It's also blocked on updating Zebra's librustzcash dependencies.

Prepare for the Release

• Make sure there has been at least one successful full sync test since the last state change, or start a manual full sync.
• Make sure the PRs with the new checkpoint hashes and missed dependencies are already merged.
  (See the release ticket checklist for details)

Summarise Release Changes

These steps can be done a few days before the release, in the same PR:

Change Log

Important: Any merge into main deletes any edits to the draft changelog.
Once you are ready to tag a release, copy the draft changelog into CHANGELOG.md.

We use the Release Drafter workflow to automatically create a draft changelog. We follow the Keep a Changelog format.

To create the final change log:

• Copy the latest draft changelog into CHANGELOG.md (there can be multiple draft releases)
• Delete any trivial changes
  • Put the list of deleted changelog entries in a PR comment to make reviewing easier
• Combine duplicate changes
• Edit change descriptions so they will make sense to Zebra users
• Check the category for each change
  • Prefer the "Fix" category if you're not sure

README

README updates can be skipped for urgent releases.

Update the README to:

• Remove any "Known Issues" that have been fixed since the last release.
• Update the "Build and Run Instructions" with any new dependencies.
  Check for changes in the Dockerfile since the last tag: git diff <previous-release-tag> docker/Dockerfile.
• If Zebra has started using newer Rust language features or standard library APIs, update the known working Rust version in the README, book, and Cargo.tomls

You can use a command like:

fastmod --fixed-strings '1.58' '1.65'

Create the Release PR

• Push the updated changelog and README into a new branch
  for example: bump-v1.0.0 - this needs to be different to the tag name
• Create a release PR by adding &template=release-checklist.md to the comparing url (Example).
• Freeze the batched queue using Mergify.
• Mark all the release PRs as Critical priority, so they go in the urgent Mergify queue.
• Mark all non-release PRs with do-not-merge, because Mergify checks approved PRs against every commit, even when a queue is frozen.

Update Versions and End of Support

Update Zebra Version

Choose a Release Level

Zebra follows semantic versioning. Semantic versions look like: MAJOR.MINOR.PATCH[-TAG.PRE-RELEASE]

Choose a release level for zebrad. Release levels are based on user-visible changes from the changelog:

• Mainnet Network Upgrades are major releases
• significant new features or behaviour changes; changes to RPCs, command-line, or configs; and deprecations or removals are minor releases
• otherwise, it is a patch release

Zebra's Rust API doesn't have any support or stability guarantees, so we keep all the zebra-* and tower-* crates on a beta pre-release version.

Update Crate Versions

If you're publishing crates for the first time, log in to crates.io,
and make sure you're a member of owners group.

Check that the release will work:

• Update crate versions, commit the changes to the release branch, and do a release dry-run:

# Update everything except for alpha crates and zebrad:
cargo release version --verbose --execute --allow-branch '*' --workspace --exclude zebrad --exclude zebra-scan --exclude zebra-grpc beta
# Due to a bug in cargo-release, we need to pass exact versions for alpha crates:
cargo release version --verbose --execute --allow-branch '*' --package zebra-scan 0.1.0-alpha.4
cargo release version --verbose --execute --allow-branch '*' --package zebra-grpc 0.1.0-alpha.2
# Update zebrad:
cargo release version --verbose --execute --allow-branch '*' --package zebrad patch # [ major | minor | patch ]
# Continue with the release process:
cargo release replace --verbose --execute --allow-branch '*' --package zebrad
cargo release commit --verbose --execute --allow-branch '*'

Crate publishing is automatically checked in CI using "dry run" mode, however due to a bug in cargo-release we need to pass exact versions to the alpha crates:

• Update zebra-scan and zebra-grpc alpha crates in the release-crates-dry-run workflow script
• Push the above version changes to the release branch.

Update End of Support

The end of support height is calculated from the current blockchain height:

• Find where the Zcash blockchain tip is now by using a Zcash explorer or other tool.
• Replace ESTIMATED_RELEASE_HEIGHT in end_of_support.rs with the height you estimate the release will be tagged.

Optional: calculate the release tagging height

• Add 1152 blocks for each day until the release
• For example, if the release is in 3 days, add 1152 * 3 to the current Mainnet block height

Update the Release PR

• Push the version increments and the release constants to the release branch.

Publish the Zebra Release

Create the GitHub Pre-Release

• Wait for all the release PRs to be merged
• Create a new release using the draft release as a base, by clicking the Edit icon in the draft release
• Set the tag name to the version tag,
  for example: v1.0.0
• Set the release to target the main branch
• Set the release title to Zebra followed by the version tag,
  for example: Zebra 1.0.0
• Replace the prepopulated draft changelog in the release description with the final changelog you created;
  starting just after the title ## [Zebra ... of the current version being released,
  and ending just before the title of the previous release.
• Mark the release as 'pre-release', until it has been built and tested
• Publish the pre-release to GitHub using "Publish Release"
• Delete all the draft releases from the list of releases

Test the Pre-Release

• Wait until the Docker binaries have been built on main, and the quick tests have passed:
  • ci-unit-tests-docker.yml
  • ci-integration-tests-gcp.yml
• Wait until the pre-release deployment machines have successfully launched

Publish Release

• Publish the release to GitHub by disabling 'pre-release', then clicking "Set as the latest release"

Publish Crates

• Run cargo login
• Run cargo clean in the zebra repo (optional)
• Publish the crates to crates.io: cargo release publish --verbose --workspace --execute
• Check that Zebra can be installed from crates.io:
  cargo install --locked --force --version 1.minor.patch zebrad && ~/.cargo/bin/zebrad
  and put the output in a comment on the PR.

Publish Docker Images

• Wait for the the Docker images to be published successfully.
• Un-freeze the batched queue using Mergify.
• Remove do-not-merge from the PRs you added it to

Release Failures

If building or running fails after tagging:

Tag a new release, following these instructions...

1. Fix the bug that caused the failure
2. Start a new patch release
3. Skip the Release Preparation, and start at the Release Changes step
4. Update CHANGELOG.md with details about the fix
5. Follow the release checklist for the new Zebra version

[arya2]

Omitted:

• fix(tests): Address a port conflict issue in the non-blocking logger test (#8658)
• build(deps): bump docker/build-push-action from 6.3.0 to 6.4.1 in the devops group (#8698)
• build(deps): bump the devops group across 1 directory with 4 updates (#8670)
• build(deps): bump the devops group with 2 updates (#8720)
• build(deps): bump the prod group across 1 directory with 11 updates (#8664)
• build(deps): bump the prod group across 1 directory with 15 updates (#8692)
• build(deps): bump the prod group across 1 directory with 8 updates (#8717)
• build(deps): bump the prod group with 5 updates (#8726)
• change(tests): Adds timeouts tonic clients in tests (#8724)
• chore(lints): Fixes Clippy warnings (#8723)
• fix(tests): Address a port conflict issue in the regtest_submit_blocks test (#8665)
• tests(scanner): Move zebra scanner tests to binary (#8659)