feat(components): add EnhancedStreamableTool implementation#740
Open
zhuangjie1125 wants to merge 22 commits intoalpha/08from
Open
feat(components): add EnhancedStreamableTool implementation#740zhuangjie1125 wants to merge 22 commits intoalpha/08from
zhuangjie1125 wants to merge 22 commits intoalpha/08from
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## alpha/08 #740 +/- ##
===========================================
Coverage ? 79.67%
===========================================
Files ? 146
Lines ? 15851
Branches ? 0
===========================================
Hits ? 12629
Misses ? 2241
Partials ? 981 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
zhuangjie1125
changed the title
zhuangjie1125
changed the title
zhuangjie1125
force-pushed
the
feat/infer_enhanced_stream
branch
from
1ba8a0e to
43a59c0
Compare
zhuangjie1125
changed the title
zhuangjie1125
force-pushed
the
feat/infer_enhanced_stream
branch
from
43a59c0 to
7491552
Compare
…722) * feat: add enhanced tool support with multimodal output capabilities and improve message formatting This commit introduces enhanced tool interfaces that support structured multimodal outputs, enabling tools to return rich content beyond simple text responses. Key Changes: 1. New Enhanced Tool Interfaces: - Added EnhancedInvokableTool and EnhancedStreamableTool interfaces for multimodal tool execution - Both interfaces use ToolCallInfo as input and return ToolResult for structured output 2. ToolResult Schema: - Introduced ToolResult type to represent multimodal tool outputs - Supports multiple content types: text, image, audio, video, and file - Added ToolOutputPart with Index field for streaming chunk merging - Implemented ToMessageInputParts() for seamless model integration 3. ToolsNode Enhancements: - Extended ToolsNode to support both legacy and enhanced tool types - Added automatic conversion between invokable and streamable endpoints - Implemented middleware support for enhanced tools - Enhanced interrupt and rerun mechanism to handle ToolResult 4. React Agent Integration: - Introduce enhancedToolResultSender and enhancedStreamToolResultSender types - Support sending *schema.ToolResult with multimodal content (images, audio, video, files) - Implement EnhancedInvokable and EnhancedStreamable middleware in tool result collector 5. Message.String() Enhancement: - Add formatting support for UserInputMultiContent, AssistantGenMultiContent, and MultiContent - Implement formatInputPart, formatOutputPart, and formatChatMessagePart helper functions - Create mediaPartFormatter interface with wrapper types for unified media formatting 6. User Input Multi-Content Concatenation: - Implement concatUserMultiContent function for merging MessageInputPart slices - Support text and base64 audio merging with proper MIME type handling - Integrate into ConcatMessages function 7. Callback System: - Added CallbackInput and CallbackOutput types for tool callbacks - Implemented conversion functions for different callback input/output types 8. Comprehensive Test Coverage: - Added tests for enhanced invokable and streamable tools - Added TestMessageString with 14 test cases covering various message types Impact: - Enables tools to return rich multimodal content (images, audio, video, files) - Provides foundation for more sophisticated tool implementations - Maintains full backward compatibility with existing tool ecosystem
…string]bool (#737) - Update ToolsConfig.ReturnDirectly type - Update ChatModelAgentContext.ReturnDirectly type - Update internal types: chatModelAgentExecCtx, execContext, reactConfig - Update all related function signatures - Update test files with new type syntax Change-Id: I7d819f1c44da91b76cf9a9f867a88008068153b8
…AfterModelRewriteState (#745)
shentongmartin
force-pushed
the
alpha/08
branch
from
8b3de42 to
b1fbde2
Compare
…nd reduction packages (#749)
Change-Id: I61cd3709e78b5e1ef1fe169572913e7e35946d56
* feat(adk): tool reduction middleware * refactor(adk): move ancient reduction middleware to internal package * chore(adk): reduction mw add i18n, rename TokenCounter * chore(adk): add reduction mw comments * chore(adk): refactor reduction mw config field, tool stream copy
…to value types and added extensive test suite for backend_inmemory GrepRaw implementation (#750) * feat(adk): enhance GrepRequest with value-type parameters and comprehensive test coverage Enhanced GrepRequest API by converting pointer parameters to value types and added extensive test suite for backend_inmemory GrepRaw implementation. Key Improvements: 1. GrepRequest Parameter Enhancement (backend.go): - Converted AfterLines, BeforeLines, ContextLines, HeadLimit, Offset from *int to int - Simplified parameter usage: direct value assignment instead of pointer references - Added clear semantics: values <= 0 indicate unset/default state - Improved API ergonomics and reduced boilerplate code 2. Backend_inmemory GrepRaw Support (backend_inmemory.go): - Updated parameter handling logic to support value-based parameters - Enhanced applyContext function with proper value validation (> 0 checks) - Improved applyPagination to handle int parameters correctly - Maintained backward compatibility while simplifying implementation 3. Middleware Integration (middlewares/filesystem/filesystem.go): - Added seamless pointer-to-value conversion for parameter passing - Implemented proper default value handling for HeadLimit and Offset - Ensured compatibility between middleware and backend layers 4. Comprehensive Test Suite (backend_inmemory_test.go): - Added 40+ table-driven test scenarios for GrepRaw functionality - Implemented concurrent safety tests (50 goroutines) - Added edge case tests (long lines, unicode, special characters) - Created detailed content validation tests - Added 4 performance benchmark tests - Included complete test coverage documentation Test Coverage: - All output modes: 100% (files_with_matches, content, count) - Parameter combinations: comprehensive coverage - Context lines: all combinations tested - Pagination: HeadLimit + Offset scenarios - Error handling: invalid patterns, boundary conditions - Concurrency: thread-safety verified - Performance: benchmarked common use cases Impact: - Simplified API: ContextLines: 1 instead of &contextLines - Better usability: no pointer variable creation needed - Enhanced reliability: comprehensive test coverage added - Improved maintainability: clear parameter semantics Files Changed: - adk/filesystem/backend.go: parameter type definitions - adk/filesystem/backend_inmemory.go: GrepRaw implementation - adk/filesystem/backend_inmemory_test.go: comprehensive test suite - adk/middlewares/filesystem/filesystem.go: middleware integration - adk/middlewares/filesystem/filesystem_test.go: test updates - adk/middlewares/filesystem/prompt.go: minor adjustments * feat(adk): add jsonschema descriptions to filesystem tool arguments Add detailed JSON Schema descriptions to all filesystem tool argument structs for better AI model understanding and tool invocation. Changes: - readFileArgs: Added descriptions for file_path, offset, and limit fields - writeFileArgs: Added descriptions for file_path and content fields - editFileArgs: Added descriptions for file_path, old_string, new_string, and replace_all fields with default value annotation - globArgs: Added descriptions for pattern and path fields - grepArgs: Added comprehensive descriptions for all 14 fields including pattern, path, glob, output_mode (with enum values), context options (-A/-B/-C), line numbers, case sensitivity, file type, pagination (head_limit/offset), and multiline mode Also updated tool descriptions in prompt.go: - WriteFileToolDesc: Improved usage guidelines with clearer instructions - GlobToolDesc: Simplified description focusing on key capabilities This enhancement enables LLM-based agents to better understand tool parameters and generate more accurate tool calls.
… filesystem.Backend interface (#753)
- Add EnhancedStreamFunc and OptionableEnhancedStreamFunc type definitions - Implement InferEnhancedStreamTool and InferOptionableEnhancedStreamTool functions - Implement NewEnhancedStreamTool and newOptionableEnhancedStreamTool functions - Add enhancedStreamableTool struct implementing tool.EnhancedStreamableTool interface - Add unit tests for all new functions~
zhuangjie1125
force-pushed
the
feat/infer_enhanced_stream
branch
from
7491552 to
345abd1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.