change(state): Support in-place disk format upgrades for major database version bumps

[upbqdn]

Motivation

Close #8745.

Solution

• Try to move an existing cached state to a new location so it can be used again.

Tests

• I manually tested the solution.

PR Author's Checklist

• The PR name will make sense to users.
• The PR provides a CHANGELOG summary.
• The solution is tested.
• The documentation is up to date.
• The PR has a priority label.

PR Reviewer's Checklist

• The PR Author's checklist is complete.
• The PR resolves the issue.

[arya2]

This is looking great, thank you!

[conradoplg]

There is a small fix required for Windows.

I was confused by some of the code but I think I got it. I suggested some comments which should hopefully help whoever reads the code in the future, feel free to tweak them.

Also I noticed the upgrade itself is not here, was it planned to be done in another PR?

[upbqdn]

I was confused by some of the code but I think I got it. I suggested some comments which should hopefully help whoever reads the code in the future, feel free to tweak them.

I originally had similar comments in the code, but I deleted them because I thought the info and warn messages were descriptive enough. Thanks for adding them.

[upbqdn]

Also I noticed the upgrade itself is not here, was it planned to be done in another PR?

Yes, this PR only moves the old db to a new path. Zebra then uses the db and rewrites the ValueBalance, stored under the same key as the original one. Current Zebra can read both the original and new format of ValueBalance from the db, but Zebra 1.8 can't, so we needed to bump the major db version. If we didn't move the db, Zebra 1.9 would start syncing the cached state from scratch.

The new ValueBalance has 40 bytes, and the original one has 32. It was trivial to implement deserialization of both formats, so we got backward compatibility with previous Zebra versions, but previous Zebra versions are not forward-compatible with the change since they'll panic when they try to deserialize the new format.

[conradoplg]

Ah, got it, thanks. LGTM!