Skip to content

Commit 734e30c

Browse files
Copilotlpcox
Copilot
and
lpcox
committed
Add detailed technical explanation of StreamableHTTPHandler and protocol state flow
Created comprehensive guide explaining: - StreamableHTTPHandler is frontend-only (gateway side) - It translates HTTP requests to JSON-RPC messages - Messages are sent via stdio to backend process - Backend sees raw JSON-RPC, not HTTP - Protocol state (initialize/ready) is tracked by SDK Server instance - Issue: New Server instance per request = fresh protocol state Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
1 parent 7ac3b22 commit 734e30c

1 file changed

Lines changed: 330 additions & 0 deletions

File tree

‎docs/STREAMABLE_HTTP_HANDLER_EXPLAINED.md‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
# StreamableHTTPHandler: How It Works
2+
3+
This document explains where the MCP SDK's `StreamableHTTPHandler` lives, what it does, and how it communicates with backend MCP servers.
4+
5+
## TL;DR
6+
7+
- **StreamableHTTPHandler**: Frontend only (gateway side), translates HTTP ↔ JSON-RPC
8+
- **Backend**: Just a process receiving JSON-RPC via stdio, no HTTP awareness
9+
- **What backend receives**: Plain JSON-RPC messages like `{"jsonrpc":"2.0","method":"tools/call",...}`
10+
- **Protocol state**: Tracked separately on frontend (SDK Server) and backend (server code)
11+
- **The issue**: New SDK Server instance per HTTP request = fresh protocol state, even though backend connection is reused
12+
13+
## Where Does StreamableHTTPHandler Live?
14+
15+
**Answer: Frontend only (gateway side)**
16+
17+
```
18+
┌─────────────────────────────────────────┐
19+
│ Gateway Process │
20+
│ ┌────────────────────────────────────┐ │
21+
│ │ StreamableHTTPHandler (Frontend) │ │
22+
│ │ - Receives HTTP POST requests │ │
23+
│ │ - Translates to JSON-RPC │ │
24+
│ │ - Creates SDK Server instance │ │
25+
│ │ - Tracks protocol state │ │
26+
│ └────────────────────────────────────┘ │
27+
│ ↓ stdio pipes │
28+
└──────────────┼──────────────────────────┘
29+
│ JSON-RPC messages
30+
↓
31+
┌──────────────────────────────────────────┐
32+
│ Backend Process (e.g., Serena) │
33+
│ - Receives JSON-RPC via stdin │
34+
│ - Sends JSON-RPC via stdout │
35+
│ - NO awareness of HTTP │
36+
│ - NO awareness of StreamableHTTPHandler │
37+
│ - Tracks its own state machine │
38+
└──────────────────────────────────────────┘
39+
```
40+
41+
**Key Points:**
42+
- Backend is just a process that speaks JSON-RPC over stdio
43+
- Backend never sees HTTP requests, headers, or StreamableHTTPHandler
44+
- Backend has no knowledge it's behind a gateway
45+
46+
## What Does StreamableHTTPHandler Do?
47+
48+
### Primary Function: HTTP ↔ JSON-RPC Translation
49+
50+
```
51+
HTTP Request (from agent)
52+
POST /mcp/serena
53+
Body: {"jsonrpc":"2.0","method":"tools/call","params":{...}}
54+
↓
55+
StreamableHTTPHandler
56+
- Creates SDK Server instance
57+
- Parses JSON-RPC from HTTP body
58+
- Routes to SDK Server methods
59+
↓
60+
SDK Server instance
61+
- Validates protocol state (uninitialized → ready)
62+
- Formats as JSON-RPC message
63+
↓
64+
Stdio pipes to backend
65+
- Writes: {"jsonrpc":"2.0","method":"tools/call",...}
66+
↓
67+
Backend Process (Serena)
68+
- Reads JSON-RPC from stdin
69+
- Processes request
70+
- Validates its own state
71+
- Writes response to stdout
72+
↓
73+
SDK Server instance
74+
- Reads JSON-RPC response from stdio
75+
↓
76+
StreamableHTTPHandler
77+
- Translates to HTTP response
78+
↓
79+
HTTP Response (to agent)
80+
Body: {"jsonrpc":"2.0","result":{...}}
81+
```
82+
83+
## What Gets Passed to the Backend?
84+
85+
**Answer: Only JSON-RPC messages, nothing about HTTP or protocol state**
86+
87+
### Example Flow:
88+
89+
**HTTP Request 1 (initialize):**
90+
```
91+
Agent sends HTTP:
92+
POST /mcp/serena
93+
Authorization: session-123
94+
Body: {
95+
"jsonrpc": "2.0",
96+
"id": 1,
97+
"method": "initialize",
98+
"params": {"protocolVersion": "2024-11-05", ...}
99+
}
100+
101+
Gateway StreamableHTTPHandler:
102+
- Extracts session ID from Authorization header
103+
- Creates NEW SDK Server instance for this request
104+
- SDK Server state: uninitialized
105+
- Sees "initialize" method → Valid for uninitialized state
106+
- Transitions state: uninitialized → ready
107+
108+
Backend (Serena) receives via stdio:
109+
{
110+
"jsonrpc": "2.0",
111+
"id": 1,
112+
"method": "initialize",
113+
"params": {"protocolVersion": "2024-11-05", ...}
114+
}
115+
116+
Backend does NOT receive:
117+
❌ HTTP headers (Authorization, Content-Type, etc.)
118+
❌ Session ID
119+
❌ Frontend SDK protocol state
120+
❌ Any indication this came via HTTP
121+
```
122+
123+
**HTTP Request 2 (tools/call) - SAME session:**
124+
```
125+
Agent sends HTTP:
126+
POST /mcp/serena
127+
Authorization: session-123
128+
Body: {
129+
"jsonrpc": "2.0",
130+
"id": 2,
131+
"method": "tools/call",
132+
"params": {"name": "search_code", ...}
133+
}
134+
135+
Gateway StreamableHTTPHandler:
136+
- Extracts session ID: session-123 (same as before)
137+
- Backend connection: ✅ REUSED (session pool works)
138+
- Creates NEW SDK Server instance for this request ❌
139+
- SDK Server state: uninitialized ❌
140+
- Sees "tools/call" method → Invalid for uninitialized state ❌
141+
- ERROR: "method 'tools/call' is invalid during session initialization"
142+
143+
Backend (Serena) NEVER receives this request
144+
❌ Request blocked by frontend SDK protocol validation
145+
```
146+
147+
## Protocol State: Frontend vs Backend
148+
149+
This is the critical distinction:
150+
151+
### Frontend Protocol State (SDK Server)
152+
```
153+
Location: Gateway process, SDK Server instance
154+
Tracks: MCP protocol state machine
155+
States: uninitialized → initializing → ready
156+
Problem: NEW instance per HTTP request = always uninitialized
157+
```
158+
159+
### Backend Protocol State (Server Implementation)
160+
```
161+
Location: Backend process (Serena/GitHub)
162+
Tracks: Backend's own state machine
163+
GitHub: NO state validation (stateless)
164+
Serena: ENFORCES state validation (stateful)
165+
```
166+
167+
### The Disconnect:
168+
169+
```
170+
┌─────────────────────────────────────────────────────────────┐
171+
│ Gateway (Frontend) │
172+
├─────────────────────────────────────────────────────────────┤
173+
│ Request 1: │
174+
│ SDK Server instance #1 (state: uninitialized) │
175+
│ Sees: initialize → Valid → State: ready ✅ │
176+
│ Sends to backend: {"method":"initialize",...} │
177+
│ │
178+
│ Request 2 (same session): │
179+
│ SDK Server instance #2 (state: uninitialized) ❌ │
180+
│ Sees: tools/call → Invalid → ERROR ❌ │
181+
│ Backend never receives this request │
182+
└─────────────────────────────────────────────────────────────┘
183+
↓ stdio (persistent)
184+
┌─────────────────────────────────────────────────────────────┐
185+
│ Backend Process (Same process, reused ✅) │
186+
├─────────────────────────────────────────────────────────────┤
187+
│ Received Request 1: │
188+
│ {"method":"initialize",...} │
189+
│ Backend state: uninitialized → ready ✅ │
190+
│ │
191+
│ Request 2 would have been fine: │
192+
│ Backend state: still ready ✅ │
193+
│ Would process {"method":"tools/call",...} successfully │
194+
│ But frontend SDK blocked it before backend saw it ❌ │
195+
└─────────────────────────────────────────────────────────────┘
196+
```
197+
198+
## Why GitHub Works But Serena Doesn't
199+
200+
### GitHub MCP Server (Stateless)
201+
202+
**Backend code doesn't validate protocol state:**
203+
```typescript
204+
// GitHub MCP Server
205+
server.setRequestHandler(ListToolsRequestSchema, async () => {
206+
// NO state check - just process the request
207+
// Works regardless of whether initialize was called
208+
return { tools: [...] };
209+
});
210+
211+
server.setRequestHandler(CallToolRequestSchema, async (request) => {
212+
// NO state check - just execute the tool
213+
return await executeTool(request.params.name, request.params.arguments);
214+
});
215+
```
216+
217+
**Result:**
218+
```
219+
Frontend SDK Server: uninitialized (wrong) ❌
220+
Backend doesn't care: processes request anyway ✅
221+
Works through gateway: ✅
222+
```
223+
224+
### Serena MCP Server (Stateful)
225+
226+
**Backend code validates protocol state:**
227+
```python
228+
# Serena MCP Server
229+
class SerenaServer:
230+
def __init__(self):
231+
self.state = "uninitialized"
232+
233+
async def handle_initialize(self, params):
234+
self.state = "ready"
235+
return {"protocolVersion": "2024-11-05"}
236+
237+
async def list_tools(self):
238+
if self.state != "ready": # State validation
239+
raise Error("method 'tools/list' is invalid during session initialization")
240+
return {"tools": [...]}
241+
242+
async def call_tool(self, name, arguments):
243+
if self.state != "ready": # State validation
244+
raise Error("method 'tools/call' is invalid during session initialization")
245+
return await self._execute_tool(name, arguments)
246+
```
247+
248+
**Result:**
249+
```
250+
Frontend SDK Server: uninitialized (wrong) ❌
251+
Backend validates state: rejects request ❌
252+
Fails through gateway: ❌
253+
```
254+
255+
## Backend Connection vs Protocol State
256+
257+
This is crucial to understand:
258+
259+
### Backend Connection (Works Correctly ✅)
260+
```
261+
- Managed by SessionConnectionPool
262+
- One persistent stdio connection per (backend, session)
263+
- Same Docker container process
264+
- Same stdin/stdout/stderr pipes
265+
- Connection IS reused across HTTP requests ✅
266+
```
267+
268+
### Protocol State (Doesn't Persist ❌)
269+
```
270+
- Managed by SDK Server instances
271+
- New instance created per HTTP request
272+
- Each instance starts in "uninitialized" state
273+
- Protocol state NOT preserved across HTTP requests ❌
274+
```
275+
276+
### Visual:
277+
```
278+
HTTP Request 1 (Authorization: session-123)
279+
→ NEW SDK Server (state: uninitialized)
280+
→ REUSED backend connection ✅
281+
→ Same backend process ✅
282+
→ {"method":"initialize"} sent
283+
284+
HTTP Request 2 (Authorization: session-123)
285+
→ NEW SDK Server (state: uninitialized) ❌
286+
→ REUSED backend connection ✅
287+
→ Same backend process ✅
288+
→ {"method":"tools/call"} blocked by SDK ❌
289+
```
290+
291+
## The Architecture Issue
292+
293+
The SDK's `StreamableHTTPHandler` was designed for **stateless HTTP scenarios** where:
294+
- Each HTTP request is completely independent
295+
- No session state needs to persist
296+
- Backend doesn't validate protocol state
297+
298+
It doesn't support **stateful backends** where:
299+
- Protocol handshake must complete on the same session
300+
- Backend validates that initialize was called before other methods
301+
- Session state must persist across multiple HTTP requests
302+
303+
## Summary
304+
305+
**Where StreamableHTTPHandler lives:**
306+
- Frontend only (gateway process)
307+
308+
**What it does:**
309+
- Translates HTTP requests to JSON-RPC messages
310+
- Creates SDK Server instances to handle protocol
311+
- Sends JSON-RPC to backend via stdio
312+
313+
**What backend receives:**
314+
- Plain JSON-RPC messages via stdin
315+
- No HTTP, no headers, no session context
316+
- No frontend protocol state information
317+
318+
**The problem:**
319+
- ✅ Backend stdio connection properly reused
320+
- ✅ Backend process state maintained correctly
321+
- ❌ Frontend SDK Server instance recreated per request
322+
- ❌ Frontend protocol state reset to uninitialized
323+
- ✅ Stateless backends (GitHub) work because they don't care
324+
- ❌ Stateful backends (Serena) fail because they validate state
325+
326+
**The limitation:**
327+
- This is an SDK architectural pattern
328+
- StreamableHTTPHandler doesn't support session persistence
329+
- Backend connection pooling works, but SDK protocol state doesn't persist
330+
- Would require SDK changes or bypassing StreamableHTTPHandler entirely

0 commit comments

Comments
 (0)