Schelling point eval doesn't work #1533
Description
Fresh installation (d3dc890)
oaieval gpt-3.5-turbo schelling_point
dies around 580th sample with
(...)
openai.BadRequestError: Error code: 400 - {'error': {'message': "Sorry! We've encountered an issue with repetitive patterns in your prompt. Please try again with a different prompt.", 'type': 'invalid_prompt', 'param': 'prompt', 'code': None}}
I guess some prompt was accepted by the OpenAI API when the eval was developed and now it's rejected.