Function calling example notebook NoneType error #272
Description
The bug
v0.0.64 update for function calling does not seem to be working. It seems like guidance tries to pull content before the function being called.
To Reproduce
In a single python file:
import guidance
import json
from system.setup import setup_chat_llm
guidance.llm = guidance.llms.OpenAI("gpt-3.5-turbo-0613", caching=False, api_key='my_api_key')
print("running guidance version: ", guidance.__version__)
def get_current_weather(location, unit="fahrenheit"):
""" Get the current weather in a given location.
Parameters
----------
location : string
The city and state, e.g. San Francisco, CA
unit : "celsius" or "fahrenheit"
"""
weather_info = {
"location": location,
"temperature": "71",
"unit": unit,
"forecast": ["sunny", "windy"],
}
return json.dumps(weather_info)
# define a guidance program that uses tools
program = guidance("""
{{~#system~}}
You are a helpful assistant.
{{>tool_def functions=functions}}
{{~/system~}}
{{~#user~}}
Get the current weather in New York City.
{{~/user~}}
{{~#each range(10)~}}
{{~#assistant~}}
{{gen 'answer' max_tokens=50 function_call="auto"}}
{{~/assistant~}}
{{#if not callable(answer)}}{{break}}{{/if}}
{{~#function name=answer.__name__~}}
{{answer()}}
{{~/function~}}
{{~/each~}}""") # type: ignore
executed_program = program(functions=[
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
}
}
], get_current_weather=get_current_weather)
print(executed_program["answer"])The code will throw an error from _gen.py
Traceback (most recent call last):
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 109, in run
await self.visit(self.parse_tree, VariableStack([self.program._variables], self))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 524, in visit
command_output = await command_function(*positional_args, **named_args)
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/library/_each.py", line 93, in each
new_content += await parser.visit(
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 524, in visit
command_output = await command_function(*positional_args, **named_args)
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/library/_assistant.py", line 13, in assistant
return await role(role_name="assistant", hidden=hidden, _parser_context=_parser_context, **kwargs)
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/library/_role.py", line 17, in role
new_content += await parser.visit(
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 559, in visit
visited_children.append(await self.visit(child, variable_stack, inner_next_node, inner_next_next_node, inner_prev_node, node, parent_node))
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 266, in visit
visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node]
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 266, in <listcomp>
visited_children = [await self.visit(child, variable_stack, next_node, next_next_node, prev_node, node, parent_node) for child in node]
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/_program_executor.py", line 379, in visit
command_output = await command_function(*positional_args, **named_args)
File "/home/tcheng/neo2/.venv/lib/python3.10/site-packages/guidance/library/_gen.py", line 173, in gen
generated_value += new_text
TypeError: can only concatenate str (not "NoneType") to str
I've tried to see what new_text is from resp
{
"id": "<some id>",
"object": "chat.completion",
"created": 1687315088,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"New York City\"\n}"
}
},
"finish_reason": "function_call",
"text": null
}
],
"usage": {
"prompt_tokens": 154,
"completion_tokens": 18,
"total_tokens": 172
}
}
My understanding from OpenAI's API, the response need to be sent back to OpenAI's API after the function is being called locally and this does not seem to be the case here?
Thank you again so much for putting this together.
System info (please complete the following information):
- OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): WSL2
- Guidance Version (
guidance.__version__): 0.0.64