Support for Anthropic + OpenRouter

[MayankGoel28]

More support for external LLM providers or easy templating for adding such providers will be really helpful.

[jzleibo]

Feel free to add more language_model wrappers, you can add them here:

https://github.com/google-deepmind/concordia/tree/main/concordia/language_model

And add a line in the switch statement in the setup utility here: https://github.com/google-deepmind/concordia/blob/main/concordia/language_model/utils.py#L61-L80

Take a look at the other language_model wrappers to see what needs to be implemented. There are two main functions:

sample_text -- this function takes a prompt as argument and returns text with a loose expectation that the returned text should be a "completion" of the prompt. You may also want to put whatever LLM/API-specific incantations are necessary to prevent refusal to answer questions in fictional scenarios here too, and error handling / rate limiting code.

and

sample_choice -- this function takes a prompt and a set of answer choices as arguments and returns one of the choices. For open weights models, the best way to implement this is usually to concatenate the answer choices into the prompt and then compare the logits associated with completion by the different answer choices. For closed weights models, the best way to implement this is usually just to ask for a response that exactly matches one of the prompted answer choices. You need to write some code to handle the, hopefully rare, case when the model does not return exactly one of the answer choices. Take a look at the other examples to see how this is done. The together_ai wrapper is an example that uses the open weights approach and base_gpt_model is an example that uses the closed weights approach.

Once you're done, send us a pull request and we'll review it.

[jzleibo]

I've just realized that we need to think about how to maintain all these wrappers. The underlying APIs seem to change pretty rapidly. We will definitely need to figure out a way to crowdsource this maintenance work if we are to continue to support a wide variety of models. I'll add a README about this, which will hopefully encourage people to add whatever wrappers they need and maintain the existing ones when they see breakage due to underlying API changes.

[jzleibo]

I just vibe-wrote the following README using Gemini 2.5 and the info in this comment thread. Not half bad I think.

Language Model Wrappers: A Call for Community Contribution

This document outlines the process for adding and maintaining language model wrappers in Concordia. As the landscape of language model APIs is constantly evolving, we rely on the community to help us keep these wrappers up-to-date and to expand our support for a wider variety of models. Your contributions are crucial for the continued success and utility of this project.

---

Why We Need Your Help

The language_model wrappers in Concordia provide a standardized interface for interacting with various language models. However, the underlying APIs for these models can change frequently, leading to breakages in our existing wrappers. To ensure that Concordia remains a robust and reliable tool for the community, we need your help in maintaining these wrappers and adding new ones.

By contributing to this effort, you can:

• Ensure continued access to your preferred language models within Concordia.
• Expand the capabilities of Concordia by adding support for new and emerging models.
• Become an active member of the Concordia community and collaborate with other researchers and developers.

---

How to Contribute

We welcome and encourage pull requests that add new language model wrappers or fix existing ones. Here’s how you can get started:

Adding a New Language Model Wrapper

If you want to add a wrapper for a new language model, please follow these steps:

1. Create a new file in the concordia/language_model/ directory. You can use the existing wrappers as a template.
2. Implement the two main functions:
   • sample_text(prompt): This function should take a prompt string and return a text completion from the language model.
   • sample_choice(prompt, choices): This function should take a prompt and a list of choices, and return one of the choices.
3. Add a new case to the switch statement in the setup utility located at concordia/language_model/utils.py to integrate your new wrapper.
4. Submit a pull request with your changes. We will review it and merge it as soon as possible.

When implementing sample_choice, the approach often depends on whether you have access to the model's internal states (i.e., weights and logits):

• Open-Weights Approach: For models where you can access the tokenwise predicted log probabilities, the best method is to append the answer choices to the prompt and compare the log probabilities produced by the model generating each choice. This provides a more direct way of measuring the model's preference. The together_ai DefaultCompletion wrapper is a good example of this.

• Closed-Weights Approach: For models accessed via APIs that don't expose tokenwise predicted log probabilities (like many commercial models), the common approach is to ask the model in the prompt to return a response that exactly matches one of the provided choices. This requires careful parsing and error handling for cases where the model's output doesn't perfectly match one of the options. The base_gpt_model wrapper demonstrates this technique.

Maintaining Existing Wrappers

If you notice that an existing wrapper is broken due to an API change, we encourage you to fix it. You can do this by:

1. Identifying the source of the breakage in the wrapper's code.
2. Updating the code to be compatible with the new API.
3. Submitting a pull request with your fixes.

Your contributions will help to ensure that Concordia remains a valuable resource for the entire community. Thank you for your support!

[MayankGoel28]

@jzleibo I am adding functionality for huggingface models - which needs the library torch. should this go as part of the requirements.txt or as an optional install prompt if using hf?

[jzleibo]

Do the following instructions work? Maybe we'll add it to the README. Can you test it?

Any new deps need to be added to install_requires in setup.py, these dependencies have the lightest constraints that still work.

requirements.txt on the other hand is used by GitHub for testing and is very highly constrained: it pins exactly which version of every dependency to use, even the dependency's dependencies. So it means users can use pip install -r requirements.txt to obtain exactly the same dependencies as we have (and have tested) on GitHub.

We also pin store a SHA with it, the SHA hash is a security measure that ensures the pinned version really is the code that pip downloads.

So once you have a pull request with the new dep added to setup.py, you'll need to update requirements.txt so it can work for everyone else. You can use ./bin/update-requirements.sh to do this, but it updates all the deps and that might get annoying as it could surface unrelated incompatibiilties. I'm not sure how to only update new deps (dropping the --upgrade from that script perhaps).

[daksh-maheshwari-ai-enggr]

Hi, I’d like to work on this issue.

I recently implemented a similar language model wrapper (Groq) for Concordia on pr #204 , so I’m already familiar with the wrapper structure and the sample_text / sample_choice interface.

I can add support for Anthropic and OpenRouter following the same pattern.

Could you please assign this issue to me?

[daksh-maheshwari-ai-enggr]

Hi!

For the OpenRouter wrapper — since their SDK is still beta,
should we use the OpenAI-compatible client instead, or stick with the OpenRouter SDK?

[jzleibo]

Feel free to use your own judgment here. We don't have any preference between those SDKs.