Name	Name	Last commit message	Last commit date
parent directory ..
.gitignore	.gitignore
Dockerfile	Dockerfile
README.md	README.md
coherence.txt	coherence.txt
eval.py	eval.py
groundedness.txt	groundedness.txt
relevance.txt	relevance.txt
requirements.txt	requirements.txt

Name

Last commit message

Last commit date

Sample Evaluation Script

To install the required packages, run the following command:

pip install -r requirements.txt

To configure, see the below section.

To run the evaluation script, run the following command:

python eval.py

This will start listening to the defined queue.

Configuration

You should create an .env file in the root directory or set environmental variables some other way. All of these are required:

AZURE_STORAGE_CONNECTION_STRING: The connection string for the Azure Storage Account that will have the queue.
QUEUE_NAME: The name of the queue in the Azure Storage Account.
AZURE_OPENAI_API_KEY: The API key for the OpenAI API.
AZURE_OPENAI_ENDPOINT: The endpoint for the OpenAI API.
AZURE_OPENAI_DEPLOYMENT: The deployment for the OpenAI API.
CATALOG_API_ENDPOINT: The endpoint for the Catalog API. (ex. http://localhost:6030)