Fix Bedrock Converse API returning both json_tool_call and real tools when tools and response_format are used

[haggai-backline]

Summary

Fixed issue #18381 where Bedrock Converse API returns both json_tool_call (internal tool for structured output) and real tools when tools and response_format parameters are used together.

Changes

• Modified _transform_response method in litellm/llms/bedrock/chat/converse_transformation.py:

  • Added logic to detect when both json_tool_call and real tools are present in the response
  • Filter out json_tool_call when multiple tools are returned and json_mode is enabled
  • Preserve existing behavior when json_tool_call is the only tool (convert to content)
  • Only return real tools to the user, hiding the internal json_tool_call implementation detail

• Added comprehensive tests in tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py:

  • test_transform_response_with_structured_response_calling_tool: Tests scenario where only real tool is called
  • test_transform_response_with_both_json_tool_call_and_real_tool: Tests the bug scenario where both tools are returned
  • Both tests verify that json_tool_call is properly filtered out from the response

Relevant issues

Fixes #18381

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

When using both tools and response_format parameters with Bedrock Converse API, LiteLLM internally adds a fake tool called json_tool_call to handle structured output. However, Bedrock sometimes returns both:

1. The json_tool_call (internal implementation detail)
2. The real tool that the user defined

This caused the response to contain the internal json_tool_call in the tool_calls array, which should be hidden from the user.

Solution

Modified the _transform_response method to intelligently filter tool calls based on the scenario:

1. Single tool (json_tool_call only): Convert to content (existing behavior)
2. Multiple tools with json_tool_call present: Filter out json_tool_call and only return real tools
3. Multiple tools without json_tool_call: Return all tools as-is (no change)

Code Changes

File: litellm/llms/bedrock/chat/converse_transformation.py

Lines 1473-1525: Refactored tool filtering logic to handle three scenarios:

• Detect if json_tool_call is present in the response
• If it's the only tool, convert to content (existing behavior)
• If there are multiple tools, filter out json_tool_call and return only real tools
• Only set tool_calls if there are tools remaining after filtering

Testing

Added two comprehensive test cases:

1. test_transform_response_with_structured_response_calling_tool (line 623):

   • Tests scenario where both tools are provided but only the real tool is called
   • Verifies that only the real tool appears in the response

2. test_transform_response_with_both_json_tool_call_and_real_tool (line 750):

   • Tests the exact bug scenario from issue [Bug]: Bedrock returns fake json_tool_call tool when using response_format + tools together, breaking OpenAI Agents SDK #18381
   • Simulates Bedrock returning both json_tool_call and real tool
   • Verifies that json_tool_call is filtered out and only real tool is returned

All 58 tests in test_converse_transformation.py are passing.

Edge Cases Handled

• ✅ Single json_tool_call only → Convert to content
• ✅ Multiple tools with json_tool_call → Filter out json_tool_call
• ✅ Multiple tools without json_tool_call → Return all tools
• ✅ No tools → No changes
• ✅ Empty tools list → No changes

Backward Compatibility

This change is fully backward compatible:

• Existing behavior for single json_tool_call is preserved
• Only affects the new edge case where both tools are returned
• No API changes or breaking changes to existing functionality

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
litellm	Ready	Preview, Comment	Dec 23, 2025 3:01pm