about config

[Violettttee]

is the code missing the config?

and i see two datasets from the code and paper
one is : dataset = load_dataset(
"edinburgh-dawg/labelchaos", args.config, split="test", token=HF_READ_TOKEN
)

another is the dataset mentioned in this repositories.

whats the difference?

[saxenarohit]

Hi,
Thanks for using MMLU-Redux!

The config refers to the subset of the dataset. For labelchaos, you can select any one of these
['bad_options_clarity', 'bad_questions_clarity', 'clean', 'clean_subsampled', 'multiple_correct_answers', 'no_correct_answer', 'small', 'wrong_groundtruth']

For information on dataset differences, please refer to the readme under Supervised Fine-tuning--> LabelChaos, where we have explained the use of the LabelChaos dataset.
https://github.com/aryopg/mmlu-redux?tab=readme-ov-file#labelchaos

Thanks.