Language Restriction for LLM Outputs

[yileitu]

I am currently working on a project involving LLMs and have a specific requirement regarding the output language. I am looking to restrict the LLM's output to a specific language, preferably English, even if the input prompt is in a different language.

For instance, my prompt is:

Frage: Roger hat 5 Tennisbälle. Er kauft noch 2 Dosen Tennisbälle. In jeder Dose sind 3 Tennisbälle. Wie viele Tennisbälle hat er jetzt?

Let's think step by step.

Here, I asked a math reasoning question in German, and I provided an English Chain of Thought (CoT) prompting, expecting the LLM to respond in English. However, in practice, I have noticed that the language of the response can be random.

My questions are as follows:

• Does Guidance currently support restricting the LLM's output language? If so, any example code?
• If this feature is not supported at the moment, are there any plans to include it in the future?
• If Guidance does not support this feature, are there any other library that implement this functionality?
  Any suggestions would be greatly appreciated.

Best wishes to Guidance team.

[nidhi-kulhari]

did you find a solution for this?

[hudson-ai]

In general, guidance and related libraries work with formal grammars, e.g. regular expressions and context-free-grammars. Natural human languages do not fit these molds very well (although some extremely large yet incomplete attempts to build these do indeed exist). You could do something simplified like write a regular expression that restricts outputs to strings of words found in a given dictionary, but this is probably MASSIVE and potentially beyond the scale that any of these libraries could handle.

Another approach would be to ditch formal grammars altogether and set-up some kind of classifier-in-the-loop to detect the target language -- similar work has been done to prevent LMs from producing "harmful" outputs. But this is difficult to do efficiently, and I am not currently aware of any libraries that have a solution here yet.

TLDR: not feasible with existing tooling, but I'm hopeful that some better solutions to this problem may develop in the future