Skip to content

vulkan upload weights per layer, reduce peak ram usage - #6534

Merged
nihui merged 4 commits into
Tencent:masterfrom
nihui:vulkan-upload-p
Feb 5, 2026
Merged

vulkan upload weights per layer, reduce peak ram usage#6534
nihui merged 4 commits into
Tencent:masterfrom
nihui:vulkan-upload-p

Conversation

@nihui

@nihui nihui commented Feb 5, 2026

Copy link
Copy Markdown
Member

No description provided.

@github-actions github-actions Bot added the core label Feb 5, 2026
@tencent-adm

Copy link
Copy Markdown
Member

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@nihui nihui closed this Feb 5, 2026
@nihui nihui reopened this Feb 5, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request refactors Vulkan model weight uploading to reduce peak RAM usage by uploading weights layer-by-layer instead of all at once. The key optimization is chunking uploads into 256MB batches, which reduces memory pressure during model loading.

Changes:

  • Removed centralized NetPrivate::upload_model() method and moved upload logic inline into Net::load_model() to enable per-layer uploads
  • Added VkTransfer::reset() and VkTransfer::pending_upload_total() methods to support incremental uploads with periodic submission
  • Implemented upload chunking logic that submits and resets the VkTransfer command buffer after accumulating 256MB of pending uploads

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/net.cpp Moved weight upload logic inline with model loading loop; added 256MB chunking to reduce peak memory usage
src/command.h Added reset() and pending_upload_total() methods to VkTransfer class API
src/command.cpp Implemented upload tracking, reset functionality, and getter for pending upload size

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/command.cpp
Comment thread src/command.cpp
Comment thread src/net.cpp
@codecov-commenter

codecov-commenter commented Feb 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.61404% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.94%. Comparing base (e665c48) to head (53a3d99).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/command.cpp 18.51% 22 Missing ⚠️
src/net.cpp 70.00% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6534      +/-   ##
==========================================
- Coverage   92.96%   92.94%   -0.02%     
==========================================
  Files         809      809              
  Lines      256977   256643     -334     
==========================================
- Hits       238900   238546     -354     
- Misses      18077    18097      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/command.cpp
@nihui
nihui merged commit 7d5d32d into Tencent:master Feb 5, 2026
147 of 155 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants