Is LiteLLM compatible with OpenAI agents SDK?

[vlerenc]

Hi,

More a question than anything else: Since LiteLLM appears "compatible" with OpenAI APIs, is it compatible with the new Agents SDK from OpenAI: https://openai.github.io/openai-agents-python/ref.

Did anybody managed to combine both maybe already?

Their agent SDK looks pretty neat (what I would expect/built myself in the past), so I would seriously consider it, but not if I have to run against OpenAI. I use Azure and Bedrock through LiteLLM and like it a lot this way, but the agent SDK is something very interesting (simple, yet powerful).

Thanks in advance.

Cheers, Vedran

[vlerenc]

Nice, it works, but the agent SDK seems to make an unexpected call. Anyway, something to debug further, but for now, combining https://docs.litellm.ai/docs/providers/openai_compatible with https://openai.github.io/openai-agents-python/config/ let me to:

1. Launch the litellm proxy:

$ litellm --debug --config model_list.yaml

2. Try this code (I created an alias model called simple):

from agents import Agent, Runner, set_default_openai_api, set_default_openai_client
from openai import AsyncOpenAI

custom_client = AsyncOpenAI(base_url="http://0.0.0.0:4000", api_key="abc...123")
set_default_openai_client(custom_client)
set_default_openai_api("chat_completions")

agent = Agent(name="Assistant", model="simple", instructions="You are a helpful assistant")

result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")

print(result.final_output)

That's it. You see the call being logged by the litellm proxy and the code yields a haiku, but also an uncaught exception afterwards about a wrong API key even though the actual call went through:

Code calls back to self,
Layers folding, deep within,
Endless paths unfold.

Tracing client error 401: {
  "error": {
    "message": "Incorrect API key provided: abc********************123. You can find your API key at https://platform.openai.com/account/api-keys.",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_api_key"
  }
}

It appears to me (for now until I debug the matter) as if the agent SDK "calls home" or something like that, because that call is not logged with litellm. If true, that's not nice by OpenAI (in comparison, not about agents, but tools, Anthropic did MCP in the open).

[vlerenc]

They have plugged in an exit handler agents.tracing.setup.TraceProvider.shutdown(). Here the docs I found: https://openai.github.io/openai-agents-python/config/#tracing.

You can get rid of it via:

from agents import set_tracing_disabled
set_tracing_disabled(True)

The ugly way:

import atexit
atexit._clear()

Or you can configure a tracing API key via https://openai.github.io/openai-agents-python/ref/#agents.set_tracing_export_api_key, but since you cannot inject a client, it wouldn't work with litellm, I think.

However, what you can do and what probably anyway makes more sense is to configure your own TracingProcessor (thank you OpenAI for allowing to do that):

from agents import set_trace_processors
from agents.tracing.processor_interface import TracingProcessor

class MyTracingProcessor(TracingProcessor):
    def on_trace_start(self, trace: "Trace") -> None:
        print(f"on_trace_start({trace})")

    def on_trace_end(self, trace: "Trace") -> None:
        print(f"on_trace_end({trace})")

    def on_span_start(self, span: "Span[Any]") -> None:
        print(f"on_span_start({span})")

    def on_span_end(self, span: "Span[Any]") -> None:
        print(f"on_span_end({span})")

    def shutdown(self) -> None:
        print("shutdown()")

    def force_flush(self) -> None:
        print("force_flush()")

set_trace_processors([MyTracingProcessor()]) # overwrites the default calling home

[vlerenc]

OK, putting it all together now, with a custom tracing processor:

from typing import Any

from agents import Agent, Runner, set_default_openai_api, set_default_openai_client, set_trace_processors
from agents.tracing.processor_interface import TracingProcessor
from agents.tracing.spans import Span
from agents.tracing.traces import Trace
from openai import AsyncOpenAI


# define custom tracing processor;
# default is to call home to OpenAI reusing the provided
# model API key, which won't work with litellm proxy
class MyTracingProcessor(TracingProcessor):
    def on_trace_start(self, trace: Trace) -> None:
        print(f"on_trace_start({trace.export()})")

    def on_trace_end(self, trace: Trace) -> None:
        print(f"on_trace_end({trace.export()})")

    def on_span_start(self, span: Span[Any]) -> None:
        print(f"on_span_start({span.export()})")

    def on_span_end(self, span: Span[Any]) -> None:
        print(f"on_span_end({span.export()})")

    def shutdown(self) -> None:
        print("shutdown()")

    def force_flush(self) -> None:
        print("force_flush()")

# create OpenAI client running against the litellm proxy, plug it in,
# switch from new responses API to more common chat completions API,
# and replace the home-calling tracing processor with a custom one
custom_client = AsyncOpenAI(base_url="http://0.0.0.0:4000", api_key="abc...123")
set_default_openai_client(custom_client)
set_default_openai_api("chat_completions")
set_trace_processors([MyTracingProcessor()])

# create sample agent
agent = Agent(name="Assistant", model="simple", instructions="You are a helpful assistant")

# run the agent and print the response
print(Runner.run_sync(agent, "Write a haiku about recursion in programming."))

[vlerenc]

@ishaan-jaff @krrishdholakia
FYI: In case you like to give it a try with OpenAI's new agent SDK,
possibly integrate the above information into your docs if/as you see fit.

[spiritanand]

@vlerenc have you been able to add metadata/session id/user id, etc, to the agent and see those in the completion requests on the litellm dashboard?

More context - #12802

[ashwiningle6]

@vlerenc can you share the model_list.yaml for reference?