diff --git a/content/manuals/ai/model-runner/_index.md b/content/manuals/ai/model-runner/_index.md index 549575dc923..d63aa0cf2d5 100644 --- a/content/manuals/ai/model-runner/_index.md +++ b/content/manuals/ai/model-runner/_index.md @@ -39,20 +39,25 @@ Models are pulled from Docker Hub the first time they're used and stored locally ### Enable DMR in Docker Desktop -1. Navigate to the **Beta features** tab in settings. -2. Tick the **Enable Docker Model Runner** setting. -3. If you are running on Windows with a supported NVIDIA GPU, you should also see and be able to tick the **Enable GPU-backed inference** setting. +1. In the settings view, navigate to the **Beta features** tab. +1. Tick the **Enable Docker Model Runner** setting. +1. If you are running on Windows with a supported NVIDIA GPU, you should also see and be able to tick the **Enable GPU-backed inference** setting. +1. Optional: If you want to enable TCP support, select the **Enable host-side TCP support** + 1. In the **Port** field, type the port of your choice. + 1. If you are interacting with Model Runner from a local frontend web app, + in **CORS Allows Origins**, select the origins that Model Runner should accept requests from. + An origin is the URL where your web app is running, for example `http://localhost:3131`. You can now use the `docker model` command in the CLI and view and interact with your local models in the **Models** tab in the Docker Desktop Dashboard. > [!IMPORTANT] > -> For Docker Desktop versions 4.41 and earlier, this settings lived under the **Experimental features** tab on the **Features in development** page. +> For Docker Desktop versions 4.41 and earlier, this setting lived under the **Experimental features** tab on the **Features in development** page. ### Enable DMR in Docker Engine 1. Ensure you have installed [Docker Engine](/engine/install/). -2. DMR is available as a package. To install it, run: +1. DMR is available as a package. To install it, run: {{< tabs >}} {{< tab name="Ubuntu/Debian">}} @@ -73,13 +78,19 @@ You can now use the `docker model` command in the CLI and view and interact with {{< /tab >}} {{< /tabs >}} -3. Test the installation: +1. Test the installation: ```console $ docker model version $ docker model run ai/smollm2 ``` +1. Optional: To enable TCP support, set the port with the `DMR_RUNNER_PORT` environment variable. +1. Optional: If you enabled TCP support, you can configure CORS allowed origins with the `DMR_ORIGINS` environment variable. Possible values are: + - `*`: Allow all origins + - Comma-separated list of allowed origins + - When unspecified, all origins are denied. + ## Pull a model Models are cached locally. diff --git a/content/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md b/content/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md index ca2cc9f25a0..532585c4553 100644 --- a/content/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md +++ b/content/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md @@ -312,14 +312,16 @@ For more information, see [Networking](/manuals/desktop/features/networking.md#n > > For Docker Desktop versions 4.41 and earlier, some of these settings lived under the **Experimental features** tab on the **Features in development** page. -| Parameter | OS | Description | Version | -|:----------------------------|----|:--------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| -| `allowBetaFeatures` | | If `value` is set to `true`, beta features are enabled. | | -| `enableDockerAI` | | If `allowBetaFeatures` is true, setting `enableDockerAI` to `true` enables [Docker AI (Ask Gordon)](/manuals/ai/gordon/_index.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | -| `enableInference` | | If `allowBetaFeatures` is true, setting `enableInference` to `true` enables [Docker Model Runner](/manuals/ai/model-runner/_index.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | -|         `enableInferenceTCP` | | Enable host-side TCP support. This setting requires Docker Model Runner setting to be enabled first. | | -| `enableDockerMCPToolkit` | | If `allowBetaFeatures` is true, setting `enableDockerMCPToolkit` to `true` enables the [MCP toolkit feature](/manuals/ai/mcp-catalog-and-toolkit/toolkit.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | -| `allowExperimentalFeatures` | | If `value` is set to `true`, experimental features are enabled. | Docker Desktop version 4.41 and earlier | +| Parameter | OS | Description | Version | +|:-----------------------------------------------------|----|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------| +| `allowBetaFeatures` | | If `value` is set to `true`, beta features are enabled. | | +| `enableDockerAI` | | If `allowBetaFeatures` is true, setting `enableDockerAI` to `true` enables [Docker AI (Ask Gordon)](/manuals/ai/gordon/_index.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | +| `enableInference` | | If `allowBetaFeatures` is true, setting `enableInference` to `true` enables [Docker Model Runner](/manuals/ai/model-runner/_index.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | +|         `enableInferenceTCP` | | Enable host-side TCP support. This setting requires Docker Model Runner setting to be enabled first. | | +|         `enableInferenceTCPPort` | | Specifies the exposed TCP port. This setting requires Docker Model Runner setting to be enabled first. | | +|         `enableInferenceCORS` | | Specifies the allowed CORS origins. Empty string to deny all,`*` to accept all, or a list of comma-separated values. This setting requires Docker Model Runner setting to be enabled first. | | +| `enableDockerMCPToolkit` | | If `allowBetaFeatures` is true, setting `enableDockerMCPToolkit` to `true` enables the [MCP toolkit feature](/manuals/ai/mcp-catalog-and-toolkit/toolkit.md) by default. You can independently control this setting from the `allowBetaFeatures` setting. | | +| `allowExperimentalFeatures` | | If `value` is set to `true`, experimental features are enabled. | Docker Desktop version 4.41 and earlier | ### Enhanced Container Isolation diff --git a/content/manuals/security/for-admins/hardened-desktop/settings-management/settings-reference.md b/content/manuals/security/for-admins/hardened-desktop/settings-management/settings-reference.md index 0d3403a8962..4bb18edb3b8 100644 --- a/content/manuals/security/for-admins/hardened-desktop/settings-management/settings-reference.md +++ b/content/manuals/security/for-admins/hardened-desktop/settings-management/settings-reference.md @@ -795,7 +795,7 @@ third-party or unvetted plugins from being installed. - **Beta** settings in [Docker Desktop GUI](/manuals/desktop/settings-and-maintenance/settings.md) - Settings Management: `enableDockerAI` setting in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md) -### Enable host-side TCP support +#### Enable host-side TCP support | Default value | Accepted values | Format | |---------------|-----------------|----------| @@ -808,11 +808,36 @@ third-party or unvetted plugins from being installed. - **Beta** settings in [Docker Desktop GUI](/manuals/desktop/settings-and-maintenance/settings.md) - Settings Management: `enableDockerAI` setting in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md) - > [!NOTE] > > This setting requires Docker Model Runner setting to be enabled first. +##### Port + +| Default value | Accepted values | Format | +|---------------|-----------------|---------| +| 12434 | Integer | Integer | + +- **Description:** Specifies the exposed TCP port. +- **OS:** {{< badge color=blue text="All" >}} +- **Use case:** Connect to the Model Runner via TCP. +- **Configure this setting with:** + - **Beta features** settings in [Docker Desktop GUI](/manuals/desktop/settings-and-maintenance/settings.md) + - Settings Management: `enableInferenceTCP` setting in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md) + +##### CORS Allowed Origins + +| Default value | Accepted values | Format | +|---------------|---------------------------------------------------------------------------------|--------| +| Empty string | Empty string to deny all,`*` to accept all, or a list of comma-separated values | String | + +- **Description:** Specifies the allowed CORS origins. +- **OS:** {{< badge color=blue text="All" >}} +- **Use case:** Integration with a web app. +- **Configure this setting with:** + - **Beta features** settings in [Docker Desktop GUI](/manuals/desktop/settings-and-maintenance/settings.md) + - Settings Management: `enableInferenceCORS` setting in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md) + ### Enable Docker MCP Toolkit | Default value | Accepted values | Format | @@ -1127,4 +1152,4 @@ overrides are possible. - **OS:** {{< badge color=blue text="All" >}} - **Use case:** Allow users to authenticate with enterprise proxy servers that require Kerberos or NTLM. - **Configure this setting with:** - - Settings Management: `proxy.enableKerberosNtlm` in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md) \ No newline at end of file + - Settings Management: `proxy.enableKerberosNtlm` in the [`admin-settings.json` file](/manuals/security/for-admins/hardened-desktop/settings-management/configure-json-file.md)