Skip to content

feat(hugr-py): Support envelope compression #1994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Mar 18, 2025
Merged

feat(hugr-py): Support envelope compression #1994

merged 3 commits into from
Mar 18, 2025

Conversation

aborgna-q
Copy link
Collaborator

Closes #1979

  • Adds a default zstd feature to hugr-rs, so hugr-cli supports it.
  • Updates the validate conftest util to test both binary and textual encodings.

There's multiple python packages which provide bindings for zstd:

  • zstd, which provides a quite simple encode/decode API. It seems to follow zstd releases quite closely.
  • zstandard, a "rich Pythonic interface" to the zstd bindings, listed as an alternative by zstd.
  • pyzstd, similar to zstd.

I went with the former since it's the most popular one.

There's an open vuln report for this: https://osv.dev/vulnerability/PYSEC-2023-121, but it is outdated and the current version already patches it (since the underlying zstd c++ lib already fixed it).

@aborgna-q aborgna-q requested a review from a team as a code owner March 18, 2025 11:25
@aborgna-q aborgna-q requested a review from tatiana-s March 18, 2025 11:25
@codecov Codecov
Copy link

codecov bot commented Mar 18, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 61.53846% with 5 lines in your changes missing coverage. Please review.

Project coverage is 83.66%. Comparing base (53b9505) to head (4b6ddb7).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
hugr-py/src/hugr/envelope.py 58.33% 5 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1994      +/-   ##
==========================================
+ Coverage   83.65%   83.66%   +0.01%     
==========================================
  Files         209      209              
  Lines       38993    38995       +2     
  Branches    35666    35666              
==========================================
+ Hits        32620    32627       +7     
+ Misses       4533     4528       -5     
  Partials     1840     1840
Flag Coverage Δ
python 92.03% <61.53%> (∅)
rust 82.88% <ø> (∅)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tatiana-s
tatiana-s approved these changes Mar 18, 2025
View reviewed changes
Copy link
Contributor

@tatiana-s tatiana-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@aborgna-q aborgna-q added this pull request to the merge queue Mar 18, 2025
Merged via the queue into main with commit 434c563 Mar 18, 2025
24 of 25 checks passed
@aborgna-q aborgna-q deleted the ab/py-zstd branch March 18, 2025 16:03
This was referenced Mar 18, 2025
Merged
Closed
Merged
github-merge-queue bot pushed a commit that referenced this pull request Mar 21, 2025
@hugrbot @doug-q
chore: release (#2002)
c90545d 
## 🤖 New release

* `hugr-model`: 0.18.0 -> 0.18.1 (✓ API compatible changes)
* `hugr-core`: 0.15.0 -> 0.16.0 (✓ API compatible changes)
* `hugr-llvm`: 0.15.0 -> 0.16.0 (✓ API compatible changes)
* `hugr-passes`: 0.15.0 -> 0.16.0 (✓ API compatible changes)
* `hugr`: 0.15.0 -> 0.16.0 (✓ API compatible changes)
* `hugr-cli`: 0.15.0 -> 0.16.0 (✓ API compatible changes)

<details><summary><i><b>Changelog</b></i></summary><p>

## `hugr-model`

<blockquote>

##
[0.18.0](hugr-model-v0.17.1...hugr-model-v0.18.0)
- 2025-03-14

### Bug Fixes

- Hugr-model using undeclared derive_more features
([#1940](#1940))

### New Features

- *(hugr-model)* [**breaking**] Add `read_from_reader` and
`write_to_writer` for streaming reads and writes.
([#1871](#1871))
- `hugr-model` AST ([#1953](#1953))

### Refactor

- *(hugr-model)* Reexport `bumpalo` from `hugr-model`
([#1870](#1870))
</blockquote>

## `hugr-core`

<blockquote>

##
[0.16.0](hugr-core-v0.15.0...hugr-core-v0.16.0)
- 2025-03-21

### Bug Fixes

- correct `CallIndirect` tag from `FnCall` to `DataflowChild`
([#2006](#2006))
- StaticArrayValue serialisation
([#2009](#2009))

### New Features

- traits for transforming Types/TypeArgs/etc.
([#1991](#1991))
- add exit operation to prelude
([#2008](#2008))
- Add llvm codegen for collections.static_array
([#2003](#2003))
</blockquote>

## `hugr-llvm`

<blockquote>

##
[0.16.0](hugr-llvm-v0.15.0...hugr-llvm-v0.16.0)
- 2025-03-21

### Bug Fixes

- Remove return from val_or_panic
([#1999](#1999))

### New Features

- add exit operation to prelude
([#2008](#2008))
- Add llvm codegen for collections.static_array
([#2003](#2003))
</blockquote>

## `hugr-passes`

<blockquote>

##
[0.16.0](hugr-passes-v0.15.0...hugr-passes-v0.16.0)
- 2025-03-21

### Bug Fixes

- correct `CallIndirect` tag from `FnCall` to `DataflowChild`
([#2006](#2006))
</blockquote>

## `hugr`

<blockquote>

##
[0.16.0](hugr-v0.15.0...hugr-v0.16.0)
- 2025-03-21

### Bug Fixes

- correct `CallIndirect` tag from `FnCall` to `DataflowChild`
([#2006](#2006))
- StaticArrayValue serialisation
([#2009](#2009))

### New Features

- traits for transforming Types/TypeArgs/etc.
([#1991](#1991))
- add exit operation to prelude
([#2008](#2008))
- Add llvm codegen for collections.static_array
([#2003](#2003))
- *(hugr-py)* Support envelope compression
([#1994](#1994))
</blockquote>

## `hugr-cli`

<blockquote>

##
[0.16.0](hugr-cli-v0.15.0...hugr-cli-v0.16.0)
- 2025-03-21

### New Features

- *(hugr-cli)* Nicer error when passing a non-envelope file
([#2007](#2007))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: Douglas Wilson <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Mar 21, 2025
@aborgna-q @hugrbot
chore(main): release hugr-py 0.11.2 (#1996)
bf3889f 
🤖 I have created a release *beep* *boop*
---


##
[0.11.2](hugr-py-v0.11.1...hugr-py-v0.11.2)
(2025-03-21)


### Features

* add exit operation to prelude
([#2008](#2008))
([6bd7665](6bd7665))
* Add llvm codegen for collections.static_array
([#2003](#2003))
([f3dd145](f3dd145))
* **hugr-py:** Add `StaticArray` to standard extensions
([#1985](#1985))
([cf860f3](cf860f3)),
closes [#1984](#1984)
* **hugr-py:** Support envelope compression
([#1994](#1994))
([434c563](434c563))


### Bug Fixes

* StaticArrayValue serialisation
([#2009](#2009))
([3fe6bf8](3fe6bf8))


### Reverts

* Revert breaking change to StaticArrayValue
([33a2b49](33a2b49))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: hugrbot <[email protected]>
@hugrbot hugrbot mentioned this pull request Jul 3, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tatiana-s tatiana-s tatiana-s approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

hugr-py: Support zstd compression on hugr envelopes
2 participants
@aborgna-q @tatiana-s