diff --git a/content/guides/agentic-ai.md b/content/guides/agentic-ai.md new file mode 100644 index 00000000000..09da953c617 --- /dev/null +++ b/content/guides/agentic-ai.md @@ -0,0 +1,394 @@ +--- +title: Build and run agentic AI applications with Docker +linktitle: Agentic AI applications +keywords: AI, Docker, Model Runner, MCP Toolkit, Docker Offload, AI agents, application development +summary: | + Learn how to create AI agent applications using Docker Model Runner, MCP Toolkit, and Docker Offload. +params: + tags: [AI] + time: 30 minutes +--- + +## Introduction + +Agentic applications are transforming how software gets built. These apps don't +just respond, they decide, plan, and act. They're powered by models, +orchestrated by agents, and integrated with APIs, tools, and services in real +time. + +All these new agentic applications, no matter what they do, share a common +architecture. It's a new kind of stack, built from three core components: + +- Models: These are your GPTs, CodeLlamas, Mistrals. They're doing the + reasoning, writing, and planning. They're the engine behind the intelligence. + +- Agent: This is where the logic lives. Agents take a goal, break it down, and + figure out how to get it done. They orchestrate everything. They talk to the + UI, the tools, the model, and the gateway. + +- MCP gateway: This is what links your agents to the outside world, including + APIs, tools, and services. It provides a standard way for agents to call + capabilities via the Model Context Protocol (MCP). + +Docker makes this AI-powered stack simpler, faster, and more secure by unifying +models, tool gateways, and cloud infrastructure into a developer-friendly +workflow that uses Docker Compose. + +![A diagram of the agentic stack](./images/agentic-ai-diagram.webp) + +This guide walks you through the core components of agentic development and +shows how Docker ties them all together with the following tools: + +- [Docker Model Runner](../manuals/ai/model-runner/_index.md) lets you run LLMs + locally with simple command and OpenAI-compatible APIs. +- [Docker MCP Catalog and + Toolkit](../manuals/ai/mcp-catalog-and-toolkit/_index.md) helps you discover + and securely run external tools, like APIs and databases, using the Model + Context Protocol (MCP). +- [Docker MCP Gateway](/ai/mcp-gateway/) lets you orchestrate and manage MCP servers. +- [Docker Offload](/offload/) provides a powerful, GPU-accelerated + environment to run your AI applications with the same Compose-based + workflow you use locally. +- [Docker Compose](../manuals/ai/compose/model-runner.md/) is the tool that ties it all + together, letting you define and run multi-container applications with a + single file. + +For this guide, you'll start by running the app in Docker Offload, using the +same Compose workflow you're already familiar with. Then, if your machine +hardware supports it, you'll run the same app locally using the same workflow. +Finally, you'll dig into the Compose file, Dockerfile, and app to see how it all +works together. + +## Prerequisites + +To follow this guide, you need to: + + - [Install Docker Desktop 4.43 or later](../get-started/get-docker.md) + - [Enable Docker Model Runner](/ai/model-runner/#enable-dmr-in-docker-desktop) + - [Join Docker Offload Beta](/offload/quickstart/) + +## Step 1: Clone the sample application + +You'll use an existing sample application that demonstrates how to connect a +model to an external tool using Docker's AI features. + +```console +$ git clone https://github.com/docker/compose-for-agents.git +$ cd compose-for-agents/adk/ +``` + +## Step 2: Run the application with Docker Offload + +You'll start by running the application in Docker Offload, which provides a +managed environment for running AI workloads. This is ideal if you want to +leverage cloud resources or if your local machine doesn't meet the hardware +requirements to run the model locally. Docker Offload includes support for +GPU-accelerated instances, making it ideal for compute-intensive workloads like +AI model inference. + +To run the application with Docker Offload, follow these steps: + +1. Sign in to the Docker Desktop Dashboard. +2. In a terminal, start Docker Offload by running the following command: + + ```console + $ docker offload start + ``` + + When prompted, choose the account you want to use for Docker Offload and select + **Yes** when prompted **Do you need GPU support?**. + +3. In the `adk/` directory of the cloned repository, run the following command + in a terminal to build and run the application: + + ```console + $ docker compose up + ``` + + The first time you run this command, Docker pulls the model from Docker Hub, + which may take some time. + + The application is now running with Docker Offload. Note that the Compose workflow + is the same when using Docker Offload as it is locally. You define your + application in a `compose.yaml` file, and then use `docker compose up` to build + and run it. + +4. Visit [http://localhost:8080](http://localhost:8080). Enter a correct or + incorrect fact in the prompt and hit enter. An agent searches DuckDuckGo to + verify it and another agent revises the output. + + ![Screenshot of the application](./images/agentic-ai-app.png) + +5. Press ctrl-c in the terminal to stop the application when you're done. + +6. Run the following command to stop Docker Offload: + + ```console + $ docker offload stop + ``` + +## Step 3: Optional. Run the application locally + +If your machine meets the necessary hardware requirements, you can run the +entire application stack locally using Docker Compose. This lets you test the +application end-to-end, including the model and MCP gateway, without needing to +run in the cloud. This particular example uses the [Gemma 3 4B +model](https://hub.docker.com/r/ai/gemma3) with a context size of `10000`. + +Hardware requirements: + - VRAM: 3.5 GB + - Storage: 2.31 GB + +If your machine exceeds those requirements, consider running a with a larger +context size or a larger model to improve the agents performance. You can easily +update model and context size in the `compose.yaml` file. + +To run the application locally, follow these steps: + +1. In the `adk/` directory of the cloned repository, run the following command in a + terminal to build and run the application: + + ```console + $ docker compose up + ``` + + The first time you run this command, Docker pulls the + model from Docker Hub, which may take some time. + +2. Visit [http://localhost:8080](http://localhost:8080). Enter a correct or + incorrect fact in the prompt and hit enter. An agent searches DuckDuckGo to + verify it and another agent revises the output. + +3. Press ctrl-c in the terminal to stop the application when you're done. + +## Step 4: Review the application environment + +You can find the `compose.yaml` file in the `adk/` directory. Open it in a text +editor to see how the services are defined. + +```yaml {collapse=true,title=compose.yaml} +services: + adk: + build: + context: . + ports: + # expose port for web interface + - "8080:8080" + environment: + # point adk at the MCP gateway + - MCPGATEWAY_ENDPOINT=http://mcp-gateway:8811/sse + depends_on: + - mcp-gateway + models: + gemma3 : + endpoint_var: MODEL_RUNNER_URL + model_var: MODEL_RUNNER_MODEL + + mcp-gateway: + # mcp-gateway secures your MCP servers + image: docker/mcp-gateway:latest + use_api_socket: true + command: + - --transport=sse + # add any MCP servers you want to use + - --servers=duckduckgo + +models: + gemma3: + # pre-pull the model when starting Docker Model Runner + model: ai/gemma3:4B-Q4_0 + context_size: 10000 # 3.5 GB VRAM + # increase context size to handle search results + # context_size: 131000 # 7.6 GB VRAM +``` + +The app consists of three main components: + + - The `adk` service, which is the web application that runs the agentic AI + application. This service talks to the MCP gateway and model. + - The `mcp-gateway` service, which is the MCP gateway that connects the app + to external tools and services. + - The `models` block, which defines the model to use with the application. + +When you examine the `compose.yaml` file, you'll notice two notable elements for the model: + + - A service‑level `models` block in the `adk` service + - A top-level `models` block + +These two blocks together let Docker Compose automatically start and connect +your ADK web app to the specified LLM. + +> [!TIP] +> +> Looking for more models to use? Check out the [Docker AI Model +> Catalog](https://hub.docker.com/catalogs/models/). + +When examining the `compose.yaml` file, you'll notice the gateway service is a +Docker-maintained image, +[`docker/mcp-gateway:latest`](https://hub.docker.com/r/docker/agents_gateway). +This image is Docker's open source [MCP +Gateway](https://github.com/docker/docker-mcp/) that enables your application to +connect to MCP servers, which expose tools that models can call. In this +example, it uses the [`duckduckgo` MCP +server](https://hub.docker.com/mcp/server/duckduckgo/overview) to perform web +searches. + +> [!TIP] +> +> Looking for more MCP servers to use? Check out the [Docker MCP +> Catalog](https://hub.docker.com/catalogs/mcp/). + +With only a few lines of instructions in a Compose file, you're able to run and +connect all the necessary services of an agentic AI application. + +In addition to the Compose file, the Dockerfile and the +`entrypoint.sh` script it creates, play a role in wiring up the AI stack at build and +runtime. You can find the `Dockerfile` in the `adk/` directory. Open it in a +text editor. + +```dockerfile {collapse=true,title=Dockerfile} +# Use Python 3.11 slim image as base +FROM python:3.13-slim +ENV PYTHONUNBUFFERED=1 + +RUN pip install uv + +WORKDIR /app +# Install system dependencies +COPY pyproject.toml uv.lock ./ +RUN --mount=type=cache,target=/root/.cache/uv \ + UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy \ + uv pip install --system . +# Copy application code +COPY agents/ ./agents/ +RUN python -m compileall -q . + +COPY <