Skip to content

test: Better python roundtrip tests, and lots of fixes #2436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

aborgna-q
Copy link
Collaborator

@aborgna-q aborgna-q commented Jul 11, 2025

The python validation tests did a roundtrip serialization check by encoding the HUGR, loading it, encoding it again, and checking for differences in the serialization.
This ignored any information loss in the first encoding, and resulted in a lot of hidden bugs.

This PR changes the test in conftest.py to instead compare the original HUGR and the loaded one directly, using a "node hash" that computes the main properties of each node and its children in an index-independent way. (we do not traverse graph edges to avoid having a graph isomorphism problem).
The node hash is defined as follows, and should be easily extensible if needed. We compare hugrs by checking the hashes of their root modules.

@dataclass
class _NodeHash:
    op: str
    entrypoint: bool
    input_neighbours: int
    output_neighbours: int
    input_ports: int
    output_ports: int
    input_order_edges: int
    output_order_edges: int
    is_region: bool
    node_depth: int
    children_hashes: list[_NodeHash] # sorted
    metadata: dict[str, str]

This revealed a bunch of bugs with the json serialization, hugr builders, node iterators, ...:

  • Order edges were serialized incorrectly. The json encoding uses null for order edges, but python emitted #ports instead (and this caused problems when combined with the out port inconsistencies below).
  • Standardize string/repr formatting for specialized types and values, so e.g. after roundtriping a Some(TRUE) it still shows as Some instead of Sum(tag=1, tys=[[], [Bool]], val=[TRUE]).
  • Hugr.num_outgoing and num_incoming were counting ports instead of edges...
  • Fixed many inconsistencies with the number of output ports in FuncDefn, Case, and the likes.
  • The output port count wasn't being set in a lot of cases (this worked fine in the tests because adding an edge auto-allocates the ports).
  • Order edges were inconsistently reported in the neighbours / links iterators. I fixed it and added new tests.
  • Probably more bugfixes I'm missing.

The roundtrip checks also have options for checking all the format combinations (one variable for the encoding format and another for converting it with hugr convert before loading if necessary).
This only does json-json for the moment, as hugr-model detects multiple errors.

@aborgna-q aborgna-q requested a review from a team as a code owner July 11, 2025 16:46
@aborgna-q aborgna-q requested a review from croyzor July 11, 2025 16:46
Copy link

codecov bot commented Jul 14, 2025

Codecov Report

Attention: Patch coverage is 90.62500% with 6 lines in your changes missing coverage. Please review.

Project coverage is 82.00%. Comparing base (ec207e7) to head (49c68c0).

Files with missing lines Patch % Lines
hugr-py/src/hugr/ops.py 86.66% 4 Missing ⚠️
hugr-py/src/hugr/hugr/base.py 91.66% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2436      +/-   ##
==========================================
- Coverage   82.56%   82.00%   -0.57%     
==========================================
  Files         247      247              
  Lines       45892    45925      +33     
  Branches    41528    41528              
==========================================
- Hits        37889    37659     -230     
- Misses       5981     6244     +263     
  Partials     2022     2022              
Flag Coverage Δ
python 85.33% <90.62%> (-5.92%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aborgna-q aborgna-q requested review from ss2165 and zrho and removed request for croyzor July 14, 2025 10:07
@aborgna-q aborgna-q force-pushed the ab/python-roundtrip-tests branch from 33fc29e to f56283f Compare July 14, 2025 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant