Skip to content

Conversation

0x-fang
Copy link
Contributor

@0x-fang 0x-fang commented Aug 5, 2025

Description

This PR fixed the streaming issue with ChatBedrockConverse when input is Application Inference Profile (AIP). The issue happened because langchain-aws fails to identify the foundation model used in AIP (e.g., arn:aws:bedrock:us-east-1:111111484058:application-inference-profile/c3myu2h6fllr), therefore, it cannot set the streaming_support flag for the AIP correctly. So the entire response was returned to user as a whole rather than streaming the response back.

Solution

We have to create a Bedrock client to call Bedrock get_inference_profile control plane API, and parse the foundation model id from the response. However, the existing set_disable_streaming function works on the raw user input before the ChatBedrockConverse object is instantiated, so we have to extract the logic of determining streaming_support for models into a common function _get_streaming_support and invoke it in both places (i.e., the original place that works on raw user input, and the new place that works on the resolved model Id from get_inference_profile API call).

Issue

#538

Test

  • add new unit tests in libs/aws/tests/unit_tests/chat_models/test_bedrock_converse.py and all passed
  • run through the integration test for ChatBedrockConverse and all passed except one existing failure
    • The main branch has failed this integration test already, which is not caused by new change.
    FAILED tests/integration_tests/chat_models/test_bedrock_converse.py::test_structured_output_tool_choice_not_supported - assert 1 == 0
    

@michaelnchin
Copy link
Collaborator

Copy link
Collaborator

@michaelnchin michaelnchin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @0x-fang !

@michaelnchin michaelnchin merged commit ea4b274 into langchain-ai:main Aug 5, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Application Inference Profile (AIP) is not streaming response with ChatBedrockConverse
2 participants