Include all test formats in benchmark releases

[marioevz]

Currently, the benchmark releases only include the new Blockchain EngineX (prev Engine ReOrg) format, but this format is not easy to consume to zk provers.

We need to fill these releases including state and blockchain test formats too.

Cc @jsign

[jsign]

Including some extra explanation on how benchmark tests are consumed for zkEVM benchmarking:

• We use a tool called zkevm-benchmark-workload for the benchmarking
• This tool allows zkEVM to prove blocks coming from two main sources: RPC (if we want to benchmark mainnet/testnet/devnet chains) or tests (for EEST tests/benchmarks).
• In the EEST case, we have a script that automatically downloads the latest release. (After this issue is resolved, we have to tune a bit some stuff there)
• After the release asset is downloaded (i.e., test fixtures), then we:
  • Use reth to deserialize each fixture into blockchain state tests.
  • That BlockchainTestCase::load(...) is basically a JSON deserialization of this model.
  • After we have this deserialized, we run the test with a fork of reth, which also returns the ExecutionWitness which is required for proving. (Eventually we'll ask to merge this upstream)

After this, we have a Block and the corresponding ExecutionWitness, which contains all the pre-state, bytecodes, and MPT proof. With this information, we can statelessly run the block in a RISCV reth program that zkEVM proves.

The relevant part is that we try to leverage existing code in reth to deserialize blockchain tests, so we can generate the execution witness for proving without reinventing the wheel. The blockchain test format is very nice since each benchmark fixture is self-contained.

[marioevz]

I think the easiest way for us would be to simply include something like:

• fixtures_benchmark_30M.tar.gz, which contains the standard format as it's currently in zkevm releases
• fixtures_benchmark_grouped_pre_alloc_30M.tar.gz, which contains the new EngineX format with grouped pre-allocs

It's a simple change in .github/configs/feature.yaml and requires no modifications to the framework.

cc @danceratopz

[chfast]

I'd very much appreciate this so they can be used e.g. as evm statetest --bench test.json.

[danceratopz]

Picked this up this afternoon, will push something soonish.

[danceratopz]

• feat(fill): add --generate-all-formats to enable generation of BlockhainEngineXFixture with other formats #1855
  This change will generate all fixture fixture formats for both benchmark and normal releases.

[spencer-tb]

I'd very much appreciate this so they can be used e.g. as evm statetest --bench test.json.

My gauge is that you can't use blockchain tests? The reason I ask is due to this issue: #1770. As we can only reach benchmarking gas limits with multiple txs in Osaka.

[chfast]

I'd very much appreciate this so they can be used e.g. as evm statetest --bench test.json.

My gauge is that you can't use blockchain tests? The reason I ask is due to this issue: #1770. As we can only reach benchmarking gas limits with multiple txs in Osaka.

Blockchain tests only would be fine too.

[jsign]

@marioevz, does this mean we can create a new release? :)