server: Add model capabilities information (instruction, embedding etc.)

[0cc4m]

Name and Version

version: 8685 (0988acc)
built with GNU 13.3.0 for Linux aarch64

Operating systems

No response

Which llama.cpp modules do you know to be affected?

llama-server

Command line

Problem description & steps to reproduce

Embedding models show up in the model selection, in this case it's even the default, but the model does not work for chat. It should be hidden or shown differently.

@ggml-org/llama-webui

First Bad Commit

No response

Relevant log output

No response

[allozaur]

Okay, so a webui-only quick dirty fix would be to filter out the models that include embedding in their names, but i am not sure if it is not a bit too dirty and unreliable... @ggerganov i am not aware of any special metadata for a model to determine if it's an embedding or a completion model, so not sure if there's any quick win that we can do in this situation on the server side..?

cc @ngxson @ServeurpersoCom

[ServeurpersoCom]

Or an explicit tag to hide the models of your choice from presets.ini, like "hidden"?

[allozaur]

Well, for presets.ini i guess that it'll be enough to just add the --embedding flag to the model and we will be able to read it from /models data.

For now i am just pushing the fix that relies on heuristic checks, but i think we should revisit this and properly handle categorizing model types on the server side using metadata etc.

[0cc4m]

Usually if you do that the temporary fix sticks and the proper implementation never happens.

[ServeurpersoCom]

To be fair, this heuristic is pretty lightweight and might actually be good enough long term. The CLI flag check (--embedding) is reliable, and the name regex is just a safety net. Worst case a false negative means an embedding model shows up in chat, which is the current behavior anyway. A full server-side solution reading GGUF general.architecture would be cleaner (there's a TODO for it at line 1015 of server-models.cpp), but this covers the real-world cases with minimal code.

[ggerganov]

Both the name regex and the CLI checks seems very hacky. Let's fix this on the server side. Though it might take a while as I am not sure what would be the best way to recognize the model capabilities.