-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Is your feature request related to a problem? Please describe.
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter
class ReportType(Enum):
REPORT_1 = "report_1"
REPORT_2 = "report_2"
@classmethod
def values(cls) -> list[str]:
return [type.value for type in cls]
class ReportConfig(BaseModel):
sheet_name: str = Field(description="Name of the sheet")
requested_start_date: date = Field(description="Start date")
requested_end_date: date = Field(description="End date")
report_type: str = Field(
description="Type of the report",
enum=ReportType.values()
)
If I call
ReportConfig.model_json_schema()
to be passed into, say, OpenAI,
tools=[
{
"type": "function",
"function": {
"name": "report_work",
"description": "Adds a report. The configuration must follow the ReportConfig schema.",
"parameters": ReportConfig.model_json_schema(),
},
}
]
I get a correct call
ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')
with the correct tool call
{'properties': {'sheet_name': {'description': 'Name of the sheet', 'title': 'Sheet Name', 'type': 'string'}, 'requested_start_date': {'description': 'Start date', 'format': 'date', 'title': 'Requested Start Date', 'type': 'string'}, 'requested_end_date': {'description': 'End date', 'format': 'date', 'title': 'Requested End Date', 'type': 'string'}, 'report_type': {'description': 'Type of the report', 'enum': ['report_1', 'report_2'], 'title': 'Report Type', 'type': 'string'}}, 'required': ['sheet_name', 'requested_start_date', 'requested_end_date', 'report_type'], 'title': 'ReportConfig', 'type': 'object'}
However, when I use LiteLLMModel
with a tool with the following inputs:
inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}
I get the call with
{'report_config': {'sheet_name': 'ABC', 'requested_start_date': '2024-01-01', 'requested_end_date': '2024-01-31'}}
So first, required
fields are not passed, which results in an incomplete request. Moreover, the enum
is not properly passed as get_json_schema
function in ..._hint_utils.py
actually uses strange choice
way of defining enums
Describe the solution you'd like
I want to be able to specify ReportConfig.model_json_schema()
in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool calling
Is this not possible with the current options.
At least I am not aware how to pass the schema differently
Describe alternatives you've considered
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.
Additional context
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments