This repository hosts projects and datasets created by Google DeepMind's Superhuman Reasoning team.
A suite of advanced benchmarks designed to evaluate robust mathematical reasoning in AI. Following our 2025 IMO gold medal achievement, this release includes:
-
IMO-AnswerBench: 400 challenging short-answer problems.
-
IMO-ProofBench: 60 proof-based problems vetted by experts.
-
IMO-GradingBench: A dataset of 1000 human gradings to advance automatic evaluation.
A reasoning agent powered by Gemini Deep Think that can iteratively generate, verify, and revise solutions.
This release includes prompts and outputs from Aletheia on research level math problems.
Copyright 2025 Google LLC
All software is licensed under the Apache License, Version 2.0 (Apache 2.0); you may not use this file except in compliance with the Apache 2.0 license. You may obtain a copy of the Apache 2.0 license at: https://www.apache.org/licenses/LICENSE-2.0
All other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY). You may obtain a copy of the CC-BY license at: https://creativecommons.org/licenses/by/4.0/legalcode
Unless required by applicable law or agreed to in writing, all software and materials distributed here under the Apache 2.0 or CC-BY licenses are distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the licenses for the specific language governing permissions and limitations under those licenses.
This is not an official Google product.