feat: blocks backup / restore

[n0izn0iz]

Fixes #1827

Description

• Adds a grpc service called Backup in the tendermint2 node that allows to stream blocks efficiently
  It has a single method StreamBlocks that take a start and end height. If end height is 0 it will stream to the latest height. It is disabled by default and require enabling it in the config.toml
• Adds a contribs binary named tm2backup that pulls blocks from the backup service and store them in compressed 100-blocks files. It takes a start and end height as well as supporting resuming.
  The tar format was chosen to bundle blocks since it's widely supported and efficient. The zstandard format was chosen for compression because it's fast, has a good compression ratio and is widely supported.
• Adds a restore subcommand to the gnoland binary that allows to replay blocks from a backup. It takes the options from the start subcommand as well as the backup directory and an optional end height.
  It will start at the current node height + 1.

The restore command can only restore at backupEndHeight-1 because I did not figure a way to commit block n without block n+1. I'd gladly take ideas on how to do that.

The backup is fast enough for now IMO (< 20min for test5 on my macbook) but can be optimized because it's not parallelized.
The restore bottleneck seems to be the gnovm currently but I would need to profile to be sure.

How to create a backup

• Enable the backup service in your node's config.toml

  [backup]
  
  laddr = "localhost:4242"

• (Re-)Start your node
• Run the tm2backup command

  cd contribs/tm2backup
  tm2backup -o blocks-backup -remote http://localhost:4242

  Example output:

  (...)

  

How to create a node from a backup

• Get the genesis file, for example:

  wget https://example.com/genesis.json

• Run the restore command

  gnoland restore --lazy --backup-dir ../contribs/tm2backup/blocks-backup

• Start your node

  gnoland start
  

TODO

• blocks streaming grpc service
• 100 blocks files generation command
• restore command
• find a way to restore up to backup height if possible
• tests

[Gno2D2]

🛠 PR Checks Summary

🔴 Pending initial approval by a review team member, or review from tech-staff

Manual Checks (for Reviewers):

• IGNORE the bot requirements for this PR (force green CI check)

Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🔴 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:

1. Fix any issues flagged by automated checks.
2. Follow the Contributor Checklist to ensure your PR is ready for review.
   • Add new tests, or document why they are unnecessary.
   • Provide clear examples/screenshots, if necessary.
   • Update documentation, if required.
   • Ensure no breaking changes, or include BREAKING CHANGE notes.
   • Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

• Report a bug with the bot.
• View the bot’s configuration file.

Debug

Automated Checks

Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: n0izn0iz/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🔴 Requirement not satisfied
└── 🔴 If
    ├── 🔴 Condition
    │   └── 🔴 Or
    │       ├── 🔴 At least 1 user(s) of the organization reviewed the pull request (with state "APPROVED")
    │       ├── 🔴 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🔴 Else
        └── 🔴 And
            ├── 🟢 This label is applied to pull request: review/triage-pending
            ├── 🔴 At least 1 user(s) of the team tech-staff reviewed pull request
            └── 🔴 On no pull request

Manual Checks

**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

• Any user with comment edit permission

[codecov]

Codecov Report

❌ Patch coverage is 78.26087% with 65 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
gno.land/cmd/gnoland/start.go	52.38%	16 Missing and 4 partials ⚠️
contribs/tm2backup/main.go	83.11%	10 Missing and 3 partials ⚠️
gno.land/cmd/gnoland/restore.go	80.43%	6 Missing and 3 partials ⚠️
tm2/pkg/bft/blockchain/reactor.go	77.77%	6 Missing and 2 partials ⚠️
tm2/pkg/bft/backup/backup_svc.go	86.95%	4 Missing and 2 partials ⚠️
tm2/pkg/bft/node/node.go	68.75%	4 Missing and 1 partial ⚠️
.../backup/backuppb/backuppbconnect/backup.connect.go	86.20%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[n0izn0iz]

@jefft0 done

[jefft0]

Hi @n0izn0iz . The CI checks in master are fixed now. Do you want to merge master and run the tests again?

[n0izn0iz]

it was up to date and the failing check is a flaky one I think. the gnokms merge from yesterday introduced conflicts that I will fix asap