Skip to content

feat: use annotation_ingest_id for uniqueness hash key in s3 + db#609

Merged
PALLAVIKHEDLE merged 9 commits intomainfrom
PALLAVIKHEDLE/ingest-id-hash-key
Mar 9, 2026
Merged

feat: use annotation_ingest_id for uniqueness hash key in s3 + db#609
PALLAVIKHEDLE merged 9 commits intomainfrom
PALLAVIKHEDLE/ingest-id-hash-key

Conversation

@PALLAVIKHEDLE
Copy link
Contributor

No description provided.

@PALLAVIKHEDLE PALLAVIKHEDLE requested review from manasaV3 and uermel March 5, 2026 18:35
Copy link
Contributor

@uermel uermel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you run the schema upgrade as well as part of this PR?

Note that after merging #602 into main you'll need to merge main into this first to maintain the migration order.

daniel-ji and others added 8 commits March 6, 2026 16:30
* feat: Add support/config for InstanceSegmentationMask

* chore: Add new generated code from the new config for
InstanceSegmentationMask

* feat: add instance segmentation mask class and instance creation

* chore: remove debug/test code in configuration 10441.yaml

* feat: remove the instancesegmentation mask from v1, and enhance it for
v2

* feat: add first steps for the InstanceSegmentationMask ingestion

* fix: remove bad guard and simplify it

* feat: add dedicated shape for the InstanceSegmentationMask

* fix: remove labels param to convert function of instance seg masks
this was causing all data to be one singular label

* feat: allow passing seg list into gen and v1 of instance seg mask config

* feat: Add configuration generation for InstanceSegmentationMask

* feat: Remove mask_label for InstanceSegmentationMask

* feat: add label computation and storing before configuration generation

* feat: clean schema attribute for the InstanceSegmentationMask

* feat: add tests for the instance segmentation mask

* chore: Updating schema to add InstanceSegmentationMask

* fix: change is_a property of InstanceSegmentationMask in metadata.yaml

* chore: update portal-neuroglancer to v1.7.0

* feat: change metadata storing for labels in image-label

* feat: add tests for instance segmentation mask db ingestion

* feat: add a different method for downscaling multi-label segmentation
masks

* chore: Add back missing migration script

* chore: Update schema to add InstanceSegmentationMask shape

* chore: Fix comment and poetry version

* feat: change the container for the image-label metadata

* chore: apply old formatter for the migration

* chore: remove InstanceSegmentationMask from v1 schema

* chore: remove lefovers for v1 for InstanceSegmenationMask

* chore: add back AnnotationInstanceSegmentationFile in metadata.yaml for v1.1.0

* bugfix a merge conflict

* small fixes

* fix instance seg mask downsampling; other small fixes; add instancesegmentationmask to ingestion config

* add instance segmentation mask validation tests

* fix test

* chore: Updated [rdev] values.yaml image tags to sha-d8cd1e8

* ingestion fixes

* chore: Updated [rdev] values.yaml image tags to sha-03852f4

* add object count to instance seg mask

* chore: Updated [rdev] values.yaml image tags to sha-e7e0d72

* add is_valid check to instancesegmask

* fix alembic migration versions

* reorder migrations correctly.

---------

Co-authored-by: aranega <vincent.aranega@gmail.com>
Co-authored-by: Sean Martin <martins7@tcd.ie>
Co-authored-by: czi-github-helper[bot] <+czi-github-helper[bot]@users.noreply.github.com>
Co-authored-by: uermel <uermel@czii.org>
…nt for deposition 10314 (#610)

bring config and metadata content back into alignment.
update 'stack' helm chart version

Co-authored-by: hspitzley-czi <105455169+hspitzley-czi@users.noreply.github.com>
update 'stack' helm chart version

Co-authored-by: hspitzley-czi <105455169+hspitzley-czi@users.noreply.github.com>
@PALLAVIKHEDLE PALLAVIKHEDLE force-pushed the PALLAVIKHEDLE/ingest-id-hash-key branch from 25df2e8 to ba283f2 Compare March 7, 2026 04:10
Copy link
Contributor

@uermel uermel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good now. Also tested import with this on staging, worked.

@PALLAVIKHEDLE PALLAVIKHEDLE merged commit 87ecb60 into main Mar 9, 2026
11 checks passed
@PALLAVIKHEDLE PALLAVIKHEDLE deleted the PALLAVIKHEDLE/ingest-id-hash-key branch March 9, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants