Skip to content

Commit 56bff7c

Browse files
committed
Revert "add context-size prop"
This reverts commit 9ff222a.
1 parent 9ff222a commit 56bff7c

File tree

2 files changed

+6
-22
lines changed

2 files changed

+6
-22
lines changed

content/manuals/ai/model-runner/_index.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -380,7 +380,3 @@ The Docker Model CLI currently lacks consistent support for specifying models by
380380
## Share feedback
381381
382382
Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
383-
384-
## Related pages
385-
386-
- [Use Model Runner with Compose](/manuals/compose/how-tos/model-runner.md)

content/manuals/compose/how-tos/model-runner.md

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -40,27 +40,15 @@ services:
4040
type: model
4141
options:
4242
model: ai/smollm2
43-
context-size: 1024
4443
```
4544
46-
Notice the following:
45+
Notice the dedicated `provider` attribute in the `ai_runner` service.
46+
This attribute specifies that the service is a model provider and lets you define options such as the name of the model to be used.
4747

48-
- In the `ai_runner` service:
48+
There is also a `depends_on` attribute in the `chat` service.
49+
This attribute specifies that the `chat` service depends on the `ai_runner` service.
50+
This means that the `ai_runner` service will be started before the `chat` service to allow injection of model information to the `chat` service.
4951

50-
- `provider.type`: Specifies that the service is a `model` provider.
51-
- `provider.options`: Specifies the options of the model. In our case, we want to use
52-
`ai/smollm2`, and we set the context size to 1024 tokens.
53-
54-
> [!NOTE]
55-
> Each model has its own maximum context size. When increasing the context length,
56-
> consider your hardware constraints. In general, try to use the smallest context size
57-
> possible for your use case.
58-
59-
- In the `chat` service:
60-
61-
- `depends_on` specifies that the `chat` service depends on the `ai_runner` service. The
62-
`ai_runner` service will be started before the `chat` service, to allow injection of model information to the `chat` service.
63-
6452
## How it works
6553

6654
During the `docker compose up` process, Docker Model Runner automatically pulls and runs the specified model.
@@ -73,6 +61,6 @@ In the example above, the `chat` service receives 2 environment variables prefix
7361

7462
This lets the `chat` service to interact with the model and use it for its own purposes.
7563

76-
## Related pages
64+
## Reference
7765

7866
- [Docker Model Runner documentation](/manuals/ai/model-runner.md)

0 commit comments

Comments
 (0)