Refactor: Tool registry on server

[ngxson]

Background Description

Currently, llama-server has no knowledge about tools or MCP, all the logic is implemented on webui client side. This introduce a lot of duplicated code if we want to also support MCP and tools on llama-cli

The plan is to allow user to use built-in tools and MCP tools from llama-cli by having a mcp.json file

Possible Refactor Approaches

Tool registry will allow having 2 types of tools:

• Built-in tools, served in OAI-compat schema
• MCP tools: this will reflect just the server URL, NOT a list of tools from that server. The client will be responsible for extracting tools from this server (on webui, we already had this logic; on CLI, we will need to implement this logic)

For MCP tools, we support both streaming HTTP and stdio protocols, using http://, https:// and stdio:// protocols respectively

Client call POST /tools to effectively "proxy" the request to the destination MCP server/stdio

Example of a list returned from GET /tools:

• read_file: built-in tool
• edit_file: built-in tool
• list_files: built-in tool
• huggingface-mcp: MCP server
• python-interpreter: MCP stdio

[pwilkin]

Okay, so actually I was thinking of directly listing the tools from MCP servers etc. because that allows us to unify everything behind a single framework. I've been thinking about this quite a bit and this actually eases the implementation of skills since we can add tools like skill:<skillname>:run with the skill script path. Generally using the source:toolname approach is enough to disambiguate and also helps with accidental tool conflicts. Plus this makes the whole tool gathering / tool execution phase much less tedious since the registry basically handles the request and then delegates to the proper handler (MCP / builtin / skill).

[ngxson]

Okay, so actually I was thinking of directly listing the tools from MCP servers etc.

My initial plan looks somewhat like this, but I doubt if it will require a lot of changes in web UI in the short term.

Nevertheless, I think the first step could be to implement the C++ MCP client inside the CLI first, then investigate if it's possible for web UI to reuse that later on.

Re skills, I think it can be done completely separate to MCP stuff. Skill scripts can be exposed as tools, much like built-in tools. Skills and and resources related to skills can be discovered via doing a scan inside an user-defined directory

[allozaur]

I think that from WebUI's perspective we'd simply switch from the MCP Servers selector/enabler dropdown to Tools Selector component and we'd list built-in tools in the same tree as the JSON Schema originating tools as well as MCP originating tools:

TOOLS

-- Built-in
---- read_file
---- edit_file
---- list_files

-- Hugging Face MCP
---- tool_1
---- tool_2
-- Some other MCP

...

-- JSON Schema
---- tool_x
---- tool_y

More detailed MCP configuration would be handled in the MCP section in Settings.