Add telemetry for tool seach tool use #3289
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds telemetry tracking for the tool search feature in the Anthropic Messages API integration. The telemetry captures both successful tool discoveries and error cases, providing insights into tool search usage and performance.
Changes:
- Injected
ITelemetryServiceintoAnthropicMessagesProcessorconstructor for telemetry collection - Added telemetry event on successful tool discovery that tracks the number of tools found and related metadata
- Added telemetry event on tool search errors that captures error codes and context
Comments suppressed due to low confidence (1)
src/platform/endpoint/node/messagesApi.ts:556
- The
interactionIdis set to the same value asrequestId. In the codebase,interactionIdis typically used to group multiple requests into a logical user interaction (seeIInteractionServicein src/platform/chat/common/interactionService.ts), whilerequestIdidentifies a single request. Using the same value for both reduces the ability to correlate multiple tool search invocations that occur within the same user interaction. Consider passing the interaction ID from the calling context if available, or document that this is intentional due to architectural constraints.
{ requestId: this.requestId, interactionId: this.requestId, validateOutcome: 'unknown', invokeOutcome: 'error', toolName: TOOL_SEARCH_TOOL_NAME, model: this.model, errorCode: toolSearchResult.content.error_code },
| } | ||
| */ | ||
| this.telemetryService.sendMSFTTelemetryEvent('toolSearchToolInvoked', | ||
| { requestId: this.requestId, interactionId: this.requestId, validateOutcome: 'unknown', invokeOutcome: 'success', toolName: TOOL_SEARCH_TOOL_NAME, model: this.model }, |
There was a problem hiding this comment.
The interactionId is set to the same value as requestId. In the codebase, interactionId is typically used to group multiple requests into a logical user interaction (see IInteractionService in src/platform/chat/common/interactionService.ts), while requestId identifies a single request. Using the same value for both reduces the ability to correlate multiple tool search invocations that occur within the same user interaction. Consider passing the interaction ID from the calling context if available, or document that this is intentional due to architectural constraints.
| { requestId: this.requestId, interactionId: this.requestId, validateOutcome: 'unknown', invokeOutcome: 'success', toolName: TOOL_SEARCH_TOOL_NAME, model: this.model }, | |
| { requestId: this.requestId, validateOutcome: 'unknown', invokeOutcome: 'success', toolName: TOOL_SEARCH_TOOL_NAME, model: this.model }, |
No description provided.