Skip to content

mtmd: add mtmd_context_params::warmup option#17652

Merged
ngxson merged 2 commits into
ggml-org:masterfrom
ngxson:xsn/mtmd_warmup_bool
Dec 1, 2025
Merged

mtmd: add mtmd_context_params::warmup option#17652
ngxson merged 2 commits into
ggml-org:masterfrom
ngxson:xsn/mtmd_warmup_bool

Conversation

@ngxson

@ngxson ngxson commented Dec 1, 2025

Copy link
Copy Markdown
Collaborator

Supersede #17638

If user wants to warmup with a different image size, set warmup = false, and send one image with the desired size right after mtmd initialized for manual warmup

warmup = false can be set via the existing --no-warmup CLI argument

@ngxson ngxson requested a review from ggerganov as a code owner December 1, 2025 11:40

@ggerganov ggerganov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably more like a "reserve worst case" rather than a "warmup" if I understand correctly.

Does the warmup in CLIP do any actual ggml graph computation?

@ngxson

ngxson commented Dec 1, 2025

Copy link
Copy Markdown
Collaborator Author

Does the warmup in CLIP do any actual ggml graph computation?

It doesn't run the computation, but it allocate enough memory for cgraph.

For context: Ideally, the memory will be allocated for the worst case (max image size), but this doesn't work very well as some models support maximum image size that worth up several GB of memory for cgraph. Therefore, we have to set an arbitrary "ideal case" for this, hence set_warmup_n_tokens function.

This was done mostly to improve UX as we don't either want to limit the max image size to an arbitrary number.But we should improve this in the future.

Also, during the warmup, we also scan the cgraph for ops that are not supported by GPU.

@ngxson ngxson merged commit ecf74a8 into ggml-org:master Dec 1, 2025
66 of 70 checks passed
Anico2 added a commit to Anico2/llama.cpp that referenced this pull request Jan 15, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request May 29, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
* mtmd: add mtmd_context_params::warmup option

* reuse the common_params::warmup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants