Skip to content

feat(compose): add enhanced tool interfaces with multimodal support#760

Merged
zhuangjie1125 merged 1 commit intomainfrom
feat/support_multi_tool
Feb 5, 2026
Merged

feat(compose): add enhanced tool interfaces with multimodal support#760
zhuangjie1125 merged 1 commit intomainfrom
feat/support_multi_tool

Conversation

@zhuangjie1125
Copy link
Contributor

Introduce EnhancedInvokableTool and EnhancedStreamableTool interfaces that support returning structured multimodal results (ToolResult) instead of plain strings.

Key changes:

  1. New Tool Interfaces (components/tool/interface.go):

    • EnhancedInvokableTool: accepts ToolArgument, returns *ToolResult
    • EnhancedStreamableTool: accepts ToolArgument, returns StreamReader[*ToolResult]
  2. Tool Utils (components/tool/utils/):

    • Added InferEnhancedTool and InferEnhancedStreamTool for type inference
    • Added NewEnhancedTool and NewEnhancedStreamTool factory functions
    • Support both regular and optionable variants
  3. ToolsNode Enhancement (compose/tool_node.go):

    • Added EnhancedInvokableToolMiddleware and EnhancedStreamableToolMiddleware
    • Auto-conversion between enhanced invokable and streamable endpoints
    • Priority given to enhanced tools when both interfaces are implemented
    • Updated interrupt/rerun state to track enhanced tool results separately
  4. Schema Extensions (schema/message.go):

    • Added ToolPartType enum (text, image, audio, video, file)
    • Added ToolArgument and ToolResult structs for structured I/O
    • Added ToolOutputPart with multimodal content support
    • Added ConcatToolResults for stream chunk merging
    • Added ToMessageInputParts() for ToolResult to Message conversion
    • Enhanced Message.String() to display multimodal content
  5. React Agent Integration (flow/agent/react/):

    • Added enhanced tool result senders for MessageFuture
    • Added middleware support for enhanced tool result collection

This enables tools to return rich content like images, audio, video, and files alongside text, supporting advanced multimodal AI agent scenarios.

Introduce EnhancedInvokableTool and EnhancedStreamableTool interfaces that support
returning structured multimodal results (ToolResult) instead of plain strings.

Key changes:

1. New Tool Interfaces (components/tool/interface.go):
   - EnhancedInvokableTool: accepts ToolArgument, returns *ToolResult
   - EnhancedStreamableTool: accepts ToolArgument, returns StreamReader[*ToolResult]

2. Tool Utils (components/tool/utils/):
   - Added InferEnhancedTool and InferEnhancedStreamTool for type inference
   - Added NewEnhancedTool and NewEnhancedStreamTool factory functions
   - Support both regular and optionable variants

3. ToolsNode Enhancement (compose/tool_node.go):
   - Added EnhancedInvokableToolMiddleware and EnhancedStreamableToolMiddleware
   - Auto-conversion between enhanced invokable and streamable endpoints
   - Priority given to enhanced tools when both interfaces are implemented
   - Updated interrupt/rerun state to track enhanced tool results separately

4. Schema Extensions (schema/message.go):
   - Added ToolPartType enum (text, image, audio, video, file)
   - Added ToolArgument and ToolResult structs for structured I/O
   - Added ToolOutputPart with multimodal content support
   - Added ConcatToolResults for stream chunk merging
   - Added ToMessageInputParts() for ToolResult to Message conversion
   - Enhanced Message.String() to display multimodal content

5. React Agent Integration (flow/agent/react/):
   - Added enhanced tool result senders for MessageFuture
   - Added middleware support for enhanced tool result collection

This enables tools to return rich content like images, audio, video, and files
alongside text, supporting advanced multimodal AI agent scenarios.
@codecov
Copy link

codecov bot commented Feb 5, 2026

Codecov Report

❌ Patch coverage is 56.94864% with 285 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.47%. Comparing base (e328752) to head (49099bc).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
schema/message.go 56.69% 93 Missing and 30 partials ⚠️
compose/tool_node.go 77.43% 36 Missing and 15 partials ⚠️
components/tool/utils/invokable_func.go 0.00% 49 Missing ⚠️
components/tool/utils/streamable_func.go 54.34% 17 Missing and 4 partials ⚠️
flow/agent/react/react.go 9.09% 20 Missing ⚠️
flow/agent/react/option.go 45.16% 15 Missing and 2 partials ⚠️
components/tool/callback_extra.go 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #760      +/-   ##
==========================================
- Coverage   81.70%   80.47%   -1.23%     
==========================================
  Files         129      129              
  Lines       12427    13007     +580     
==========================================
+ Hits        10153    10468     +315     
- Misses       1518     1737     +219     
- Partials      756      802      +46     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zhuangjie1125 zhuangjie1125 merged commit 32fc2fb into main Feb 5, 2026
17 of 19 checks passed
@zhuangjie1125 zhuangjie1125 deleted the feat/support_multi_tool branch February 5, 2026 03:54
shentongmartin pushed a commit that referenced this pull request Feb 5, 2026
…760)

Introduce EnhancedInvokableTool and EnhancedStreamableTool interfaces that support
returning structured multimodal results (ToolResult) instead of plain strings.

Key changes:

1. New Tool Interfaces (components/tool/interface.go):
   - EnhancedInvokableTool: accepts ToolArgument, returns *ToolResult
   - EnhancedStreamableTool: accepts ToolArgument, returns StreamReader[*ToolResult]

2. Tool Utils (components/tool/utils/):
   - Added InferEnhancedTool and InferEnhancedStreamTool for type inference
   - Added NewEnhancedTool and NewEnhancedStreamTool factory functions
   - Support both regular and optionable variants

3. ToolsNode Enhancement (compose/tool_node.go):
   - Added EnhancedInvokableToolMiddleware and EnhancedStreamableToolMiddleware
   - Auto-conversion between enhanced invokable and streamable endpoints
   - Priority given to enhanced tools when both interfaces are implemented
   - Updated interrupt/rerun state to track enhanced tool results separately

4. Schema Extensions (schema/message.go):
   - Added ToolPartType enum (text, image, audio, video, file)
   - Added ToolArgument and ToolResult structs for structured I/O
   - Added ToolOutputPart with multimodal content support
   - Added ConcatToolResults for stream chunk merging
   - Added ToMessageInputParts() for ToolResult to Message conversion
   - Enhanced Message.String() to display multimodal content

5. React Agent Integration (flow/agent/react/):
   - Added enhanced tool result senders for MessageFuture
   - Added middleware support for enhanced tool result collection

This enables tools to return rich content like images, audio, video, and files
alongside text, supporting advanced multimodal AI agent scenarios.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants