Skip to content

Self-contained parquet export #2500

@manuelwedler

Description

@manuelwedler

Following #1668 I think we can have a separate export where sources are joined before the export and each partition is made of "self-contained" data. E.g. under /1/0xab/0xab.part1.parquet you'll find the columns of this contract joined on the contract_deployments table. So that someone can fetch a single row of this partition, that is all information about this contract, with a parquet tool, or just the whole partition.

Originally posted by @kuzdogan in #2441

As the comment above suggests, we should consider an additional export, where each file contains all data related to the included contracts. This should make it possible to get all data for one contract without having to join over multiple parquet files.

Also see verifier-alliance/parquet-export#10

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions