Cannot use streaming with Amazon Nova models through ChatBedrock when tools are bound

[Freybsnapsoft]

When tools are bound to Amazon Nova models through ChatBedrock class or bedrock: provider prefix, they return responses in one chunk when using .stream.
Using ChatBedrockConverse class or bedrock_converse: provider prefix works correctly.

Example code to reproduce:
❌ Returns response in one chunk

from langchain_aws import ChatBedrock
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"
    
llm = ChatBedrock(
    model="us.amazon.nova-pro-v1:0",
).bind_tools([get_weather])

for message_chunk in llm.stream( 
    [
        {"role": "user", "content": f"How's the weather in San Francisco"}
    ],
):
    if message_chunk.content:
        print(message_chunk.content, flush=True)

✅ Streams response

from langchain_aws import ChatBedrockConverse
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"
    
llm = ChatBedrockConverse(
    model="us.amazon.nova-pro-v1:0",
).bind_tools([get_weather])

for message_chunk in llm.stream( 
    [
        {"role": "user", "content": f"How's the weather in San Francisco"}
    ],
):
    if message_chunk.content:
        print(message_chunk.content, flush=True)

The root cause of the problem is that when using Amazon Nova models through ChatBedrock or bedrock: provider prefix, LangChain forces them to use the Converse API.

langchain-aws/libs/aws/langchain_aws/chat_models/bedrock.py

Lines 695 to 702 in d97e1b3

	@model_validator(mode="before")
	@classmethod
	def set_beta_use_converse_api(cls, values: Dict) -> Any:
	model_id = values.get("model_id", values.get("model"))
	if model_id and "beta_use_converse_api" not in values:
	values["beta_use_converse_api"] = "nova" in model_id
	return values

When calling stream on the model, the _as_converse creates a new ChatBedrockConverse class and sets the provider to an empty string - since self.provider is None.

langchain-aws/libs/aws/langchain_aws/chat_models/bedrock.py

Lines 1198 to 1211 in d97e1b3

	return ChatBedrockConverse(
	client=self.client,
	model=self.model_id,
	region_name=self.region_name,
	credentials_profile_name=self.credentials_profile_name,
	aws_access_key_id=self.aws_access_key_id,
	aws_secret_access_key=self.aws_secret_access_key,
	aws_session_token=self.aws_session_token,
	config=self.config,
	provider=self.provider or "",
	base_url=self.endpoint_url,
	guardrail_config=(self.guardrails if self._guardrails_enabled else None), # type: ignore[call-arg]
	**kwargs,
	)

This makes ChatBedrockConverse.set_disable_streaming set the disable_streaming property to False.

langchain-aws/libs/aws/langchain_aws/chat_models/bedrock_converse.py

Line 496 in d97e1b3

def set_disable_streaming(cls, values: Dict) -> Any:

Unfortunatelly, explicitely setting beta_use_converse_api to False does not solve the issue, it gives an error:

from langchain_aws import ChatBedrock
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get weather for a given city."""
    return f"It's always sunny in {city}!"
    
llm = ChatBedrock(
    model="us.amazon.nova-pro-v1:0",
    beta_use_converse_api=False,
).bind_tools([get_weather])

for message_chunk in llm.stream( 
    [
        {"role": "user", "content": f"How's the weather in San Francisco"}
    ],
):
    if message_chunk.content:
        print(message_chunk.content, flush=True)

‼️ botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModelWithResponseStream operation: Malformed input request: #: required key [messages] not found, please reformat your input and try again.

[0x-fang]

taking a look at this issue.