Skip to content

Commit e647b18

Browse files
lucas-koehlerlucas-koehler
authored
ai: Support prompt fragments and recursive variable resolving (#15196)
fixes #14899 fixes #15000 * Add prompt template AI variable Adds a new AI variable that resolves registered prompt templates based on their ID. The variable resolves to the resolved prompt template. I.e. variables and functions in the template are already resolved in the returned value. In the chat a prompt template with ID myprompt is referenced as #prompt:myprompt Adds prompt template id autocompletion for the chat window that triggers on entering the variable separator `:` and automatically after the `#prompt` variable has been autocompleted. * Resolve functions after variables in prompt service Functions must not be immediately resolved in prompt fragments because function objects are lost this way. Instead, they are left in and resolved when the final chat message or prompt containing a prompt fragment is resolved. For prompts, all variables must be resolved before resolving functions to allow resolving functions in resolved variables. - Resolve functions after variables when getting a prompt - Extend PromptService with getPromptFragment method that resolves variables but not functions - ChatRequestParser can now handle chat and prompt function formats - Add unit test for prompt service testing resolving a function within a variable * Resolve functions in resolved variables in chat request parser - Move variable resolvement from chat service to chat request parser - Resolve functions in resolved variable texts - Add unit test for this * Support explicit AI variable dependencies and recursive resolvement - Add interface AIVariableResolverWithVariableDependencies for variable resolvers that want to resolve dependencies. The variable service hands in a resolve method to these resolvers. - The AI variable service now recursively resolves all variables while providing cycle detection. If a cycle is found, the resolvement of only the recursive branch is stopped.
1 parent 6df4a06 commit e647b18

File tree

11 files changed

+891
-85
lines changed

11 files changed

+891
-85
lines changed

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,11 @@
1212
- [core] migration from deprecated `phosphorJs` to actively maintained fork `Lumino` [#14320](https://github.com/eclipse-theia/theia/pull/14320) - Contributed on behalf of STMicroelectronics
1313
Adopters importing `@phosphor` packages now need to import from `@lumino`. CSS selectors refering to `.p-` classes now need to refer to `.lm-` classes. There are also minor code adaptations, for example now using `iconClass` instead of `icon` in Lumino commands.
1414

15+
<a name="breaking_changes_1.60.0">[Breaking Changes:](#breaking_changes_1.60.0)</a>
16+
17+
- [ai-chat] `ParsedChatRequest.variables` is now `ResolvedAIVariable[]` instead of a `Map<string, AIVariable>` [#15196](https://github.com/eclipse-theia/theia/pull/15196)
18+
- [ai-chat] `ChatRequestParser.parseChatRequest` is now asynchronous and expects an additional `ChatContext` parameter [#15196](https://github.com/eclipse-theia/theia/pull/15196)
19+
1520
## 1.59.0 - 02/27/2025
1621

1722
- [ai] added claude sonnet 3.7 to default models [#15023](https://github.com/eclipse-theia/theia/pull/15023)

‎packages/ai-chat/src/common/chat-request-parser.spec.ts‎

Lines changed: 95 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -18,32 +18,42 @@ import * as sinon from 'sinon';
1818
import { ChatAgentServiceImpl } from './chat-agent-service';
1919
import { ChatRequestParserImpl } from './chat-request-parser';
2020
import { ChatAgentLocation } from './chat-agents';
21-
import { ChatRequest } from './chat-model';
21+
import { ChatContext, ChatRequest } from './chat-model';
2222
import { expect } from 'chai';
23-
import { DefaultAIVariableService, ToolInvocationRegistry, ToolInvocationRegistryImpl } from '@theia/ai-core';
23+
import { AIVariable, DefaultAIVariableService, ResolvedAIVariable, ToolInvocationRegistryImpl, ToolRequest } from '@theia/ai-core';
24+
import { ILogger, Logger } from '@theia/core';
25+
import { ParsedChatRequestTextPart, ParsedChatRequestVariablePart } from './parsed-chat-request';
2426

2527
describe('ChatRequestParserImpl', () => {
2628
const chatAgentService = sinon.createStubInstance(ChatAgentServiceImpl);
2729
const variableService = sinon.createStubInstance(DefaultAIVariableService);
28-
const toolInvocationRegistry: ToolInvocationRegistry = sinon.createStubInstance(ToolInvocationRegistryImpl);
29-
const parser = new ChatRequestParserImpl(chatAgentService, variableService, toolInvocationRegistry);
30+
const toolInvocationRegistry = sinon.createStubInstance(ToolInvocationRegistryImpl);
31+
const logger: ILogger = sinon.createStubInstance(Logger);
32+
const parser = new ChatRequestParserImpl(chatAgentService, variableService, toolInvocationRegistry, logger);
3033

31-
it('parses simple text', () => {
34+
beforeEach(() => {
35+
// Reset our stubs before each test
36+
sinon.reset();
37+
});
38+
39+
it('parses simple text', async () => {
3240
const req: ChatRequest = {
3341
text: 'What is the best pizza topping?'
3442
};
35-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
43+
const context: ChatContext = { variables: [] };
44+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
3645
expect(result.parts).to.deep.contain({
3746
text: 'What is the best pizza topping?',
3847
range: { start: 0, endExclusive: 31 }
3948
});
4049
});
4150

42-
it('parses text with variable name', () => {
51+
it('parses text with variable name', async () => {
4352
const req: ChatRequest = {
4453
text: 'What is the #best pizza topping?'
4554
};
46-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
55+
const context: ChatContext = { variables: [] };
56+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
4757
expect(result).to.deep.contain({
4858
parts: [{
4959
text: 'What is the ',
@@ -59,11 +69,12 @@ describe('ChatRequestParserImpl', () => {
5969
});
6070
});
6171

62-
it('parses text with variable name with argument', () => {
72+
it('parses text with variable name with argument', async () => {
6373
const req: ChatRequest = {
6474
text: 'What is the #best:by-poll pizza topping?'
6575
};
66-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
76+
const context: ChatContext = { variables: [] };
77+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
6778
expect(result).to.deep.contain({
6879
parts: [{
6980
text: 'What is the ',
@@ -79,11 +90,12 @@ describe('ChatRequestParserImpl', () => {
7990
});
8091
});
8192

82-
it('parses text with variable name with numeric argument', () => {
93+
it('parses text with variable name with numeric argument', async () => {
8394
const req: ChatRequest = {
8495
text: '#size-class:2'
8596
};
86-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
97+
const context: ChatContext = { variables: [] };
98+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
8799
expect(result.parts[0]).to.contain(
88100
{
89101
variableName: 'size-class',
@@ -92,11 +104,12 @@ describe('ChatRequestParserImpl', () => {
92104
);
93105
});
94106

95-
it('parses text with variable name with POSIX path argument', () => {
107+
it('parses text with variable name with POSIX path argument', async () => {
96108
const req: ChatRequest = {
97109
text: '#file:/path/to/file.ext'
98110
};
99-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
111+
const context: ChatContext = { variables: [] };
112+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
100113
expect(result.parts[0]).to.contain(
101114
{
102115
variableName: 'file',
@@ -105,16 +118,82 @@ describe('ChatRequestParserImpl', () => {
105118
);
106119
});
107120

108-
it('parses text with variable name with Win32 path argument', () => {
121+
it('parses text with variable name with Win32 path argument', async () => {
109122
const req: ChatRequest = {
110123
text: '#file:c:\\path\\to\\file.ext'
111124
};
112-
const result = parser.parseChatRequest(req, ChatAgentLocation.Panel);
125+
const context: ChatContext = { variables: [] };
126+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
113127
expect(result.parts[0]).to.contain(
114128
{
115129
variableName: 'file',
116130
variableArg: 'c:\\path\\to\\file.ext'
117131
}
118132
);
119133
});
134+
135+
it('resolves variable and extracts tool functions from resolved variable', async () => {
136+
// Set up two test tool requests that will be referenced in the variable content
137+
const testTool1: ToolRequest = {
138+
id: 'testTool1',
139+
name: 'Test Tool 1',
140+
handler: async () => undefined
141+
};
142+
const testTool2: ToolRequest = {
143+
id: 'testTool2',
144+
name: 'Test Tool 2',
145+
handler: async () => undefined
146+
};
147+
// Configure the tool registry to return our test tools
148+
toolInvocationRegistry.getFunction.withArgs(testTool1.id).returns(testTool1);
149+
toolInvocationRegistry.getFunction.withArgs(testTool2.id).returns(testTool2);
150+
151+
// Set up the test variable to include in the request
152+
const testVariable: AIVariable = {
153+
id: 'testVariable',
154+
name: 'testVariable',
155+
description: 'A test variable',
156+
};
157+
// Configure the variable service to return our test variable
158+
// One tool reference uses chat format and one uses prompt format because the parser needs to handle both.
159+
variableService.getVariable.withArgs(testVariable.name).returns(testVariable);
160+
variableService.resolveVariable.withArgs(
161+
{ variable: testVariable.name, arg: 'myarg' },
162+
sinon.match.any
163+
).resolves({
164+
variable: testVariable,
165+
arg: 'myarg',
166+
value: 'This is a test with ~testTool1 and **~{testTool2}** and more text.',
167+
});
168+
169+
// Create a request with the test variable
170+
const req: ChatRequest = {
171+
text: 'Test with #testVariable:myarg'
172+
};
173+
const context: ChatContext = { variables: [] };
174+
175+
// Parse the request
176+
const result = await parser.parseChatRequest(req, ChatAgentLocation.Panel, context);
177+
178+
// Verify the variable part contains the correct properties
179+
expect(result.parts.length).to.equal(2);
180+
expect(result.parts[0] instanceof ParsedChatRequestTextPart).to.be.true;
181+
expect(result.parts[1] instanceof ParsedChatRequestVariablePart).to.be.true;
182+
const variablePart = result.parts[1] as ParsedChatRequestVariablePart;
183+
expect(variablePart).to.have.property('resolution');
184+
expect(variablePart.resolution).to.deep.equal({
185+
variable: testVariable,
186+
arg: 'myarg',
187+
value: 'This is a test with ~testTool1 and **~{testTool2}** and more text.',
188+
} satisfies ResolvedAIVariable);
189+
190+
// Verify both tool functions were extracted from the variable content
191+
expect(result.toolRequests.size).to.equal(2);
192+
expect(result.toolRequests.has(testTool1.id)).to.be.true;
193+
expect(result.toolRequests.has(testTool2.id)).to.be.true;
194+
195+
// Verify the result contains the tool requests returned by the registry
196+
expect(result.toolRequests.get(testTool1.id)).to.deep.equal(testTool1);
197+
expect(result.toolRequests.get(testTool2.id)).to.deep.equal(testTool2);
198+
});
120199
});

‎packages/ai-chat/src/common/chat-request-parser.ts‎

Lines changed: 68 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
import { inject, injectable } from '@theia/core/shared/inversify';
2323
import { ChatAgentService } from './chat-agent-service';
2424
import { ChatAgentLocation } from './chat-agents';
25-
import { ChatRequest } from './chat-model';
25+
import { ChatContext, ChatRequest } from './chat-model';
2626
import {
2727
chatAgentLeader,
2828
chatFunctionLeader,
@@ -35,15 +35,17 @@ import {
3535
ParsedChatRequest,
3636
ParsedChatRequestPart,
3737
} from './parsed-chat-request';
38-
import { AIVariable, AIVariableService, ToolInvocationRegistry, ToolRequest } from '@theia/ai-core';
38+
import { AIVariable, AIVariableService, createAIResolveVariableCache, getAllResolvedAIVariables, ToolInvocationRegistry, ToolRequest } from '@theia/ai-core';
39+
import { ILogger } from '@theia/core';
3940

4041
const agentReg = /^@([\w_\-\.]+)(?=(\s|$|\b))/i; // An @-agent
4142
const functionReg = /^~([\w_\-\.]+)(?=(\s|$|\b))/i; // A ~ tool function
43+
const functionPromptFormatReg = /^\~\{\s*(.*?)\s*\}/i; // A ~{} prompt-format tool function
4244
const variableReg = /^#([\w_\-]+)(?::([\w_\-_\/\\.:]+))?(?=(\s|$|\b))/i; // A #-variable with an optional : arg (#file:workspace/path/name.ext)
4345

4446
export const ChatRequestParser = Symbol('ChatRequestParser');
4547
export interface ChatRequestParser {
46-
parseChatRequest(request: ChatRequest, location: ChatAgentLocation): ParsedChatRequest;
48+
parseChatRequest(request: ChatRequest, location: ChatAgentLocation, context: ChatContext): Promise<ParsedChatRequest>;
4749
}
4850

4951
function offsetRange(start: number, endExclusive: number): OffsetRange {
@@ -57,10 +59,48 @@ export class ChatRequestParserImpl {
5759
constructor(
5860
@inject(ChatAgentService) private readonly agentService: ChatAgentService,
5961
@inject(AIVariableService) private readonly variableService: AIVariableService,
60-
@inject(ToolInvocationRegistry) private readonly toolInvocationRegistry: ToolInvocationRegistry
62+
@inject(ToolInvocationRegistry) private readonly toolInvocationRegistry: ToolInvocationRegistry,
63+
@inject(ILogger) private readonly logger: ILogger
6164
) { }
6265

63-
parseChatRequest(request: ChatRequest, location: ChatAgentLocation): ParsedChatRequest {
66+
async parseChatRequest(request: ChatRequest, location: ChatAgentLocation, context: ChatContext): Promise<ParsedChatRequest> {
67+
// Parse the request into parts
68+
const { parts, toolRequests } = this.parseParts(request, location);
69+
70+
// Resolve all variables and add them to the variable parts.
71+
// Parse resolved variable texts again for tool requests.
72+
// These are not added to parts as they are not visible in the initial chat message.
73+
// However, they need to be added to the result to be considered by the executing agent.
74+
const variableCache = createAIResolveVariableCache();
75+
for (const part of parts) {
76+
if (part instanceof ParsedChatRequestVariablePart) {
77+
const resolvedVariable = await this.variableService.resolveVariable(
78+
{ variable: part.variableName, arg: part.variableArg },
79+
context,
80+
variableCache
81+
);
82+
if (resolvedVariable) {
83+
part.resolution = resolvedVariable;
84+
// Resolve tool requests in resolved variables
85+
this.parseFunctionsFromVariableText(resolvedVariable.value, toolRequests);
86+
} else {
87+
this.logger.warn(`Failed to resolve variable ${part.variableName} for ${location}`);
88+
}
89+
}
90+
}
91+
92+
// Get resolved variables from variable cache after all variables have been resolved.
93+
// We want to return all recursilvely resolved variables, thus use the whole cache.
94+
const resolvedVariables = await getAllResolvedAIVariables(variableCache);
95+
96+
return { request, parts, toolRequests, variables: resolvedVariables };
97+
}
98+
99+
protected parseParts(request: ChatRequest, location: ChatAgentLocation): {
100+
parts: ParsedChatRequestPart[];
101+
toolRequests: Map<string, ToolRequest>;
102+
variables: Map<string, AIVariable>;
103+
} {
64104
const parts: ParsedChatRequestPart[] = [];
65105
const variables = new Map<string, AIVariable>();
66106
const toolRequests = new Map<string, ToolRequest>();
@@ -72,7 +112,7 @@ export class ChatRequestParserImpl {
72112

73113
if (previousChar.match(/\s/) || i === 0) {
74114
if (char === chatFunctionLeader) {
75-
const functionPart = this.tryParseFunction(
115+
const functionPart = this.tryToParseFunction(
76116
message.slice(i),
77117
i
78118
);
@@ -107,8 +147,7 @@ export class ChatRequestParserImpl {
107147
if (i !== 0) {
108148
// Insert a part for all the text we passed over, then insert the new parsed part
109149
const previousPart = parts.at(-1);
110-
const previousPartEnd =
111-
previousPart?.range.endExclusive ?? 0;
150+
const previousPartEnd = previousPart?.range.endExclusive ?? 0;
112151
parts.push(
113152
new ParsedChatRequestTextPart(
114153
offsetRange(previousPartEnd, i),
@@ -131,8 +170,25 @@ export class ChatRequestParserImpl {
131170
)
132171
);
133172
}
173+
return { parts, toolRequests, variables };
174+
}
175+
176+
/**
177+
* Parse text for tool requests and add them to the given map
178+
*/
179+
private parseFunctionsFromVariableText(text: string, toolRequests: Map<string, ToolRequest>): void {
180+
for (let i = 0; i < text.length; i++) {
181+
const char = text.charAt(i);
134182

135-
return { request, parts, toolRequests, variables };
183+
// Check for function markers at start of words
184+
if (char === chatFunctionLeader) {
185+
const functionPart = this.tryToParseFunction(text.slice(i), i);
186+
if (functionPart) {
187+
// Add the found tool request to the given map
188+
toolRequests.set(functionPart.toolRequest.id, functionPart.toolRequest);
189+
}
190+
}
191+
}
136192
}
137193

138194
private tryToParseAgent(
@@ -201,8 +257,9 @@ export class ChatRequestParserImpl {
201257
return new ParsedChatRequestVariablePart(varRange, name, variableArg);
202258
}
203259

204-
private tryParseFunction(message: string, offset: number): ParsedChatRequestFunctionPart | undefined {
205-
const nextFunctionMatch = message.match(functionReg);
260+
private tryToParseFunction(message: string, offset: number): ParsedChatRequestFunctionPart | undefined {
261+
// Support both the and chat and prompt formats for functions
262+
const nextFunctionMatch = message.match(functionPromptFormatReg) || message.match(functionReg);
206263
if (!nextFunctionMatch) {
207264
return;
208265
}

‎packages/ai-chat/src/common/chat-service.ts‎

Lines changed: 5 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ import {
3535
ChatContext,
3636
} from './chat-model';
3737
import { ChatRequestParser } from './chat-request-parser';
38-
import { ParsedChatRequest, ParsedChatRequestAgentPart, ParsedChatRequestVariablePart } from './parsed-chat-request';
38+
import { ParsedChatRequest, ParsedChatRequestAgentPart } from './parsed-chat-request';
3939

4040
export interface ChatRequestInvocation {
4141
/**
@@ -191,7 +191,9 @@ export class ChatServiceImpl implements ChatService {
191191
}
192192
session.title = request.text;
193193

194-
const parsedRequest = this.chatRequestParser.parseChatRequest(request, session.model.location);
194+
const resolutionContext: ChatSessionContext = { model: session.model };
195+
const resolvedContext = await this.resolveChatContext(session.model.context.getVariables(), resolutionContext);
196+
const parsedRequest = await this.chatRequestParser.parseChatRequest(request, session.model.location, resolvedContext);
195197
const agent = this.getAgent(parsedRequest, session);
196198

197199
if (agent === undefined) {
@@ -205,25 +207,9 @@ export class ChatServiceImpl implements ChatService {
205207
};
206208
}
207209

208-
const resolutionContext: ChatSessionContext = { model: session.model };
209-
const resolvedContext = await this.resolveChatContext(session.model.context.getVariables(), resolutionContext);
210210
const requestModel = session.model.addRequest(parsedRequest, agent?.id, resolvedContext);
211211
resolutionContext.request = requestModel;
212212

213-
for (const part of parsedRequest.parts) {
214-
if (part instanceof ParsedChatRequestVariablePart) {
215-
const resolvedVariable = await this.variableService.resolveVariable(
216-
{ variable: part.variableName, arg: part.variableArg },
217-
resolutionContext
218-
);
219-
if (resolvedVariable) {
220-
part.resolution = resolvedVariable;
221-
} else {
222-
this.logger.warn(`Failed to resolve variable ${part.variableName} for ${session.model.location}`);
223-
}
224-
}
225-
}
226-
227213
let resolveResponseCreated: (responseModel: ChatResponseModel) => void;
228214
let resolveResponseCompleted: (responseModel: ChatResponseModel) => void;
229215
const invocation: ChatRequestInvocation = {
@@ -259,6 +245,7 @@ export class ChatServiceImpl implements ChatService {
259245
resolutionRequests: readonly AIVariableResolutionRequest[],
260246
context: ChatSessionContext,
261247
): Promise<ChatContext> {
248+
// TODO use a common cache to resolve variables and return recursively resolved variables?
262249
const resolvedVariables = await Promise.all(
263250
resolutionRequests.map(async contextVariable => {
264251
const resolvedVariable = await this.variableService.resolveVariable(contextVariable, context);

0 commit comments

Comments
 (0)