Skip to content

Commit 4c31218

Browse files
authored
[Misc] remove --model from vllm serve usage (vllm-project#17944)
Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>
1 parent 6831189 commit 4c31218

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

examples/online_serving/openai_chat_completion_client_with_tools.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,12 @@
77
templates, or your own - the model default doesn't work for tool calls with vLLM
88
See the vLLM docs on OpenAI server & tool calling for more details.
99
10-
vllm serve --model mistralai/Mistral-7B-Instruct-v0.3 \
10+
vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
1111
--chat-template examples/tool_chat_template_mistral.jinja \
1212
--enable-auto-tool-choice --tool-call-parser mistral
1313
1414
OR
15-
vllm serve --model NousResearch/Hermes-2-Pro-Llama-3-8B \
15+
vllm serve NousResearch/Hermes-2-Pro-Llama-3-8B \
1616
--chat-template examples/tool_chat_template_hermes.jinja \
1717
--enable-auto-tool-choice --tool-call-parser hermes
1818
"""

0 commit comments

Comments
 (0)