Skip to content

Manage chat models

Jump to bottom
igardev edited this page Sep 15, 2025 · 4 revisions

Manage chat models

Requred servers

  • No servers required

Overview

Chat models configurations are stored and could be reused. For simplicity the term "chat models" will be used as a synonim for chat models configurations. Chat models could be for local models (run by llama-vscode) and for externally run servers. They have properties: name, local start command (llama-server command to start a server with this model locally), ai model (as required by the provider), endpoint, is key required

Chat models configurations could be added/deleted/viewed/selected/deselected/added from huggingface/exported/imported

How to use it

Select "Chat models..." from llama-vscode menu

  • Add local model
    Enter the requested properties.
    Name, local start command and endpoint are required
    Use models, which support text completion, usually with "Instruct" in the name (for example i.e. Qwen2.5-Coder-1.5B-Instruct)

  • Add external model
    Enter the requested properties.
    Name and endpoint are required. Use models, which support text completion, usually with "Instruct" in the name.

  • Delete models
    Select the model you want to delete from the list and delete it.

  • View
    Select a model from the list to view all the details for this model

  • Selected
    Select a model from the list to select it. If the model is a local one (has a command in local start command) a llama.cpp server with this model will be started. Only one chat model could be selected at a time.

  • Deselect
    Deselect the currently selected model. If the model is local, the llama.cpp server will be started.

  • Add model from huggingface
    Enter search words to find a model from huggingface. If the model is selected it will be automatically downloaded (if not yet done) and a llama.cpp server will be started with it.

  • Export
    A model could be exported as a .json files. This file could be shared with other users, modified if needed and imported again. Select a model to export it.

  • Import
    A model could be imported from a .json file - select a file to import it.

Clone this wiki locally