-
Notifications
You must be signed in to change notification settings - Fork 74
docs: add chunk retrieval architecture diagrams (PE-8468) #474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
djwhitt
wants to merge
6
commits into
develop
Choose a base branch
from
PE-8468-architecture-diagrams
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #474 +/- ##
========================================
Coverage 71.16% 71.16%
========================================
Files 62 62
Lines 17514 17514
Branches 1110 1110
========================================
Hits 12463 12463
Misses 5038 5038
Partials 13 13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Add PlantUML diagrams documenting chunk retrieval architecture: - chunk-source-priority.puml: Shows fallback order of chunk sources - chunk-component-architecture.puml: Shows component relationships Update CLAUDE.md to ensure diagrams are maintained as code changes. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Double diagram size for better readability (DPI 400) - Update component names for accuracy: - "SQLite DB" → "TX Offset Index" - "File Cache" → "Local Cache" - "AR.IO Peers" → "AR.IO Network" - "Arweave" → "Arweave Network" - Add separate Browser and AR.IO Peer client components - Update abbreviations for consistency (TOI, LC, ARIO, AR) - Change "Lookup TX" to "Lookup offset" for clarity 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Rename diagram file from chunk-source-priority to chunk-retrieval-cascade - Update title to "Chunk Retrieval Cascade" for better clarity - Set DPI to 300 for improved readability - Clarify cache types: "in-memory & disk" instead of just "local disk" - Distinguish AR.IO network (frequently used) from Arweave (complete set) - Add rebroadcast flow from S3 back to Arweave network 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add new diagram showing contiguous data source hierarchy - Shows 5-tier fallback system: Local Cache → Trusted Gateways → Chunk Reconstruction → TX Data → AR.IO Network - Consistent with chunk retrieval cascade diagram naming and style - Set to 200 DPI for optimal display size - Based on actual data source configuration and retrieval flow 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…468) - Add sequence diagram showing new efficient data retrieval flow - Uses root parent offset to skip bundle hierarchy traversal - Shows offset source can be local index or network-based - Demonstrates direct chunk fetching using calculated offset + range - Includes caching and streaming response flow - Set to 200 DPI for consistent display sizing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Add alternative approach using on-demand ANS-104 header parsing - Shows root ID source providing only transaction ID (no pre-calculated offsets) - Demonstrates dynamic offset calculation by parsing bundle headers - Trades performance for reduced storage requirements - Maintains same chunk fetching and assembly flow as pre-calculated approach - Set to 200 DPI for consistent display sizing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
425352b
to
123bfe9
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Details
Creates two new architecture diagrams in
docs/diagrams/src/
:chunk-source-priority.puml
- Shows the fallback order when retrieving chunks (Local Cache → AR.IO Peers → Arweave Network → Legacy S3)chunk-component-architecture.puml
- Shows component relationships and how the Composite Source uses configurable parallelismThese diagrams follow the existing pattern in the codebase and help developers understand the chunk retrieval system architecture.
Test plan
🤖 Generated with Claude Code