Model architectures #90
Conversation
|
📦 Build Artifacts Available |
…vLLM Inference session Signed-off-by: Alexandre Marques <[email protected]>
This update introduces a new section detailing the currently supported model architectures, including Llama-3 and Qwen3, along with their trained checkpoints. This addition enhances the documentation by providing users with clear information on available models. Signed-off-by: Alexandre Marques <[email protected]>
Signed-off-by: Alexandre Marques <[email protected]>
anmarques
force-pushed
the
model_architectures
branch
from
75f13f5 to
2a13cc4
Compare
There was a problem hiding this comment.
@anmarques I'd like to adapt this further to ideally balance discoverability, clarity, and depth. Can we change this into a table format and either add it as a new sub section in either the Overview (after key features) or in Resources (### Models and then ### Research Implementations).
I'm thinking something like the following for the table format so it is compact and informative as a really quick example:
| Model / Verifier | Training & Creation | Deployment (vLLM) | Pretrained Checkpoints |
|---|---|---|---|
| Qwen3 (8B) | Eagle 3 ✔️, HASS ⏳ | Eagle 3 ✔️ | Eagle 3 ✔️ |
| Llama-3.1 (70B) | Eagle 3 ⏳ | Eagle 3 ⏳ | ✖️ |
✔️ = Supported,⏳ = In Progress,✖️ = Not Available
|
I like having this central place for everything supported, LGTM once the existing comments are resolved! |
|
I agree with the suggested changes. Rahul and I are putting together a similar table for in-depth vLLM overview that we'll add in the next week or so. |
This PR lists the model architectures currently supported and planned to be supported with the speculators format in vLLM. It specifies model architectures that are currently supported and others for which there are trained checkpoints, from Red Hat AI, EAGLE or HASS.