[Chore] Cleanup guided namespace, move to structured outputs config #22772

aarnphm · 2025-08-13T00:44:57Z

Continuation of #17420

This PR introduces the args --structured-output-config as a way to unify all related structured outputs config in one CLI field.
This would help simplify general UX for specifying custom options with backends.

I also remove all previous guided_decoding options

This would also be considered breaking. There will be no --guided-decoding-* option anymore. Instead, you should use --structured-outputs-config '{...}' or --structured-outputs-config.backend outlines

Signed-off-by: Aaron Pham [email protected]
Signed-off-by: Harry Mellor [email protected]
Co-authored-by: Nick Hill [email protected]
Co-authored-by: Harry Mellor [email protected]

github-actions · 2025-08-13T00:45:18Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

mergify · 2025-08-13T00:45:35Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @aarnphm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Aaron Pham <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

hmellor · 2025-09-17T11:43:04Z

Kernel and model executor failures appear to have come from main.

The entrypoints failure is legitimate. For some reason, tool calling is no longer using structured outputs. We can see this is the case by running pytest -vsx tests/entrypoints/openai/test_chat.py::test_named_tool_use[True].

Before this PR, the schema was included in the user message which would allow the model to cheat in the event that structured outputs stopped working.

The modification I've made to test_named_tool_use() (removing the schema from the user message) passes on main, meaning that structured outputs were being used for this test before this PR.

Signed-off-by: Harry Mellor <[email protected]>

Culprit commit: vllm-project/vllm#22772 --------- Signed-off-by: Agata Dobrzyniewicz <[email protected]>

…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]>

They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: #163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet

…2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]>

…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]>

They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: pytorch#163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet

…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]> Signed-off-by: Che Ruan <[email protected]>

They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: pytorch#163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet

simon-mo · 2025-09-22T20:34:07Z

@aarnphm can we add backward compatibility for one version so people know how to migrate?

aarnphm requested review from patrickvonplaten, hmellor, mgoin, russellb, DarkLight1337, robertgshaw2-redhat, simon-mo, njhill, WoosukKwon, ywang96, comaniac, alexm-redhat, zhuohan123 and youkaichao as code owners August 13, 2025 00:44

mergify bot added documentation Improvements or additions to documentation frontend performance Performance-related issues structured-output v1 labels Aug 13, 2025

mergify bot added the tool-calling label Aug 13, 2025

github-project-automation bot added this to Structured Output Aug 13, 2025

mergify bot added the needs-rebase label Aug 13, 2025

github-project-automation bot added this to Tool Calling Aug 13, 2025

chore: finalize cleanup from v0

69068cd

Signed-off-by: Aaron Pham <[email protected]>

aarnphm force-pushed the feat/decoding-args-rename-all branch from d3ac885 to 69068cd Compare August 13, 2025 00:47

aarnphm requested review from tlrmchlsmth, houseroad and yewentao256 as code owners August 13, 2025 00:47

hmellor added 5 commits September 17, 2025 00:25

Remove badly merged change

bd5ef94

Signed-off-by: Harry Mellor <[email protected]>

Fix opinionated backend selection part 2

8b38bc4

Signed-off-by: Harry Mellor <[email protected]>

Fix comment

76cb011

Signed-off-by: Harry Mellor <[email protected]>

Merge branch 'main' into pr/aarnphm/22772

87488f5

Signed-off-by: Harry Mellor <[email protected]>

Make failing test less flaky

f869f9c

Signed-off-by: Harry Mellor <[email protected]>

hmellor and others added 3 commits September 17, 2025 14:50

Fix structured output being enabled by response format and tool calling

ec94b4a

Signed-off-by: Harry Mellor <[email protected]>

Merge branch 'main' into feat/decoding-args-rename-all

2f2f3bc

Merge branch 'main' into feat/decoding-args-rename-all

476950a

DarkLight1337 requested a review from ApostaC as a code owner September 18, 2025 04:22

Fix test

5872fe7

Signed-off-by: Harry Mellor <[email protected]>

DarkLight1337 merged commit 29283e8 into vllm-project:main Sep 18, 2025
79 checks passed

github-project-automation bot moved this to Done in Tool Calling Sep 18, 2025

github-project-automation bot moved this to Done in Structured Output Sep 18, 2025

adobrzyn mentioned this pull request Sep 18, 2025

[BUGFIX] Fix hourly after PR#22772 vllm-project/vllm-gaudi#197

Merged

xuechendi pushed a commit to vllm-project/vllm-gaudi that referenced this pull request Sep 18, 2025

[BUGFIX] Fix hourly after PR#22772 (#197)

5a4e0ec

Culprit commit: vllm-project/vllm#22772 --------- Signed-off-by: Agata Dobrzyniewicz <[email protected]>

MengqingCao mentioned this pull request Sep 19, 2025

[CI] Upgrade vLLM to 20250919 (6d8246aa) and fix some broken issue vllm-project/vllm-ascend#2907

Merged

huydhn mentioned this pull request Sep 19, 2025

Clean up obsoleted vLLM tests pytorch/pytorch#163383

Closed

Yikun mentioned this pull request Sep 22, 2025

[Bug]: Fix vllm main issue (0922) vllm-project/vllm-ascend#3083

Open

jiqing-feng mentioned this pull request Sep 22, 2025

update guided decoding param to structured outputs huggingface/trl#4117

Open

qgallouedec mentioned this pull request Sep 22, 2025

Pin vLLM version huggingface/trl#4122

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[Chore] Cleanup guided namespace, move to structured outputs config #22772

[Chore] Cleanup guided namespace, move to structured outputs config #22772

aarnphm commented Aug 13, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Aug 13, 2025

Uh oh!

mergify bot commented Aug 13, 2025

Uh oh!

hmellor commented Sep 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

simon-mo commented Sep 22, 2025

Uh oh!

Uh oh!

Uh oh!

[Chore] Cleanup guided namespace, move to structured outputs config #22772

[Chore] Cleanup guided namespace, move to structured outputs config #22772

Conversation

aarnphm commented Aug 13, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 13, 2025

Uh oh!

mergify bot commented Aug 13, 2025

Uh oh!

hmellor commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

simon-mo commented Sep 22, 2025

Uh oh!

Uh oh!

aarnphm commented Aug 13, 2025 •

edited by github-actions bot

Loading

hmellor commented Sep 17, 2025 •

edited

Loading