These are the datasets models get scored on, which makes them unusually consequential: they shape what the field optimises for. They are also the datasets most at risk of leaking into training data.
Benchmarks in this category divide into knowledge tests, which ask whether the model knows something, and reasoning tests, which ask whether it can work something out. The distinction matters because a model can be strong on one and weak on the other, and because contamination affects them differently — a leaked knowledge question is simply answered, while a leaked reasoning problem may still require the working.
If you are using any of these to evaluate a model you are about to deploy, treat the public score as a shortlisting signal rather than a decision. Test items leak into web-scale pretraining corpora, and the only evaluation that fully resists this is one you built yourself and have never published.
| Dataset | Modality | Size | Licence | Best for |
|---|---|---|---|---|
| MMLU | Text | 15,900 questions | MIT | Multitasking abilities of large language models |
| SuperGPQA - LLM assessment on 285 disciplines | Text | 26,529 questions | ODC-By | LLM performance on multi-domain QA |
| SQuAD (Stanford Question Answering Dataset) | Text | 100,000 questions | CC BY-SA 4.0 | NLP question-answering models |
| SNLI | Text | 570,000 pairs | CC BY-SA 4.0 | Textual inference models |
| MathVision | Image | 3,040 problems | MIT | Visual mathematical reasoning in multimodal models |
| Qasper - Questions/Answers on scientific papers | Text | 5,049 questions | CC BY 4.0 | Read whole scientific articles |
| EXAMS - Multilingual school exam questions | Text | 24,000 questions | CC BY-SA 4.0 | Multilingual question answering |
| DuoRC - Questions/Answers about movie synopsis | Text | 187,213 examples | MIT | Extractive or abstractive question answering |
| SelQA | Text | 404,776 rows | Apache 2.0 | Answer-selection question answering |
| MAPS - Multilingual Agentic Performance & Security | Text | 805 tasks | MIT | Comparing AI agent performance |
MMLU
- Size
- About 15,900 questions (14,042 test) over 57 subjects; ~114,000 rows with the auxiliary training set
- Licence
- MIT
About 15,900 multiple-choice questions (14,042 in the test split) spanning 57 academic and professional subjects, from elementary mathematics to professional law. The larger 114,000-row figure includes a 99,842-question auxiliary training set drawn from RACE, ARC, OpenBookQA and MCTest. The standard broad-knowledge yardstick quoted in nearly every LLM release.
Evaluating multitasking abilities of large language models, comparing performance across models on specialised topics, and building standardised reasoning benchmarks.
Source: innovatiana.com
SuperGPQA - LLM assessment on 285 disciplines
- Size
- 26,529 multiple-choice questions across 285 graduate-level disciplines, JSONL
- Licence
- ODC-BY
26,529 multiple-choice questions in JSONL spanning 285 graduate-level disciplines, from law through to music. Far broader subject coverage than MMLU's 57 subjects, aimed at long-tail specialist knowledge.
Evaluating LLM performance on multi-domain QA, training or testing specialised academic models, and analysing robustness on questions from 285 disciplines.
Source: innovatiana.com
SQuAD (Stanford Question Answering Dataset)
- Size
- Over 100,000 question and answer pairs, in JSON format
- Licence
- CC BY-SA 4.0
Over 100,000 question-and-answer pairs in JSON, built on Wikipedia passages where each answer appears verbatim as a span in the supporting passage. The reference extractive reading-comprehension benchmark.
Training NLP question-answering models, benchmarking comprehension, fine-tuning LLMs for assistants and search, and testing answer extraction or generation.
Source: innovatiana.com
SNLI
- Size
- 570,000 phrase-premise/hypothesis pairs, JSON format
- Licence
- CC-BY-SA 4.0
About 570,000 manually labelled English premise-and-hypothesis sentence pairs in JSON, each marked entailment, contradiction or neutral. The foundational natural language inference corpus and a common sentence-embedding training signal.
Training and evaluating textual inference models, improving semantic understanding in NLP systems, and testing reasoning about logical relationships between sentences.
Source: innovatiana.com
MathVision
- Size
- 3,040 math problems with images (3,344 rows including the 304-problem testmini split), 64 MB, Parquet format
- Licence
- MIT
3,040 visual mathematics problems (plus a 304-problem testmini split) in a 64 MB Parquet file, drawn from real competitions and each paired with a contextual image, spanning 16 disciplines and five difficulty levels. Competition provenance keeps difficulty high.
Evaluating visual mathematical reasoning in multimodal models, training models to solve problems using images and text, and testing across mathematical disciplines.
Source: innovatiana.com
- context
- The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse ("Norman" comes from "Norseman") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. …
- question
- In what country is Normandy located?
- answers
- “France” at character 159 — given by 4 annotators
- question
- What is France a region of?
- is_impossible
- true — answers is empty; plausible_answers holds “Normandy”
Qasper - Questions/Answers on scientific papers
- Size
- 5,049 questions on 1,585 articles, structured JSON (text, questions, answers, annotations)
- Licence
- CC-BY 4.0
5,049 questions over 1,585 full-text NLP research papers in structured JSON, with answers and supporting evidence spans manually annotated inside each document. Built for long-document QA where evidence must be located.
Training models to read whole scientific articles to answer questions, testing Longformer or BigBird on evidence selection, and developing scientific QA systems.
Source: innovatiana.com
EXAMS - Multilingual school exam questions
- Size
- More than 24,000 questions in 16 languages and 24 subjects (136,082 rows across overlapping configurations)
- Licence
- CC BY-SA 4.0
More than 24,000 high school exam questions across 16 languages and 24 subjects, including mathematics, biology, history and geography (row counts in the Hugging Face release are higher because its configurations overlap), with some items supplying context and answer choices. Real exam material rather than crowd-written questions.
Training multilingual question answering, evaluating cross-language generalisation, and testing educational models in a realistic context.
Source: innovatiana.com
DuoRC - Questions/Answers about movie synopsis
- Size
- 187,213 examples, JSON text format, two subsets (SelfRC, ParaphraseRC)
- Licence
- MIT
187,213 JSON question-answer examples built from movie synopses drawn from both Wikipedia and IMDb, split into SelfRC and ParaphraseRC subsets. The paired parallel synopses force genuine paraphrase understanding.
Training extractive or abstractive question answering, evaluating comprehension across parallel passages, and strengthening paraphrase and generative answering.
Source: innovatiana.com
- question
- The particle size range of soil physical clay is:
- options
- A >0.04mm · B >0.07mm · C >0.01cm · D >0.05mm · E <0.01mm · F >0.02mm · G >0.03mm · H >0.01mm · I <0.01cm · J >0.06mm
- answer
- <0.01mm (E)
- discipline › field › subfield
- Engineering › Forestry Engineering › Forest Engineering
- difficulty
- easy
SelQA
- Size
- 404,776 rows across four overlapping configurations (about 7,900 answer-selection questions), 68 MB, JSON/Parquet format
- Licence
- Apache 2.0
404,776 rows across four overlapping configurations (about 7,900 questions with candidate sentences), 68 MB in JSON and Parquet, each giving a question, a context and candidate answers from which the correct one must be selected. Answer selection rather than span extraction.
Training answer-selection question answering, evaluating identification of the correct response in context, and improving QA for virtual assistants.
Source: innovatiana.com
MAPS - Multilingual Agentic Performance & Security
- Size
- 805 tasks, each in English and 11 translations (9,660 JSON instances)
- Licence
- MIT
805 agent tasks assembled from GAIA, MATH, SWE-Bench and ASB, each in English and 11 translations (9,660 JSON instances in all), for testing both the capability and the security of AI agents outside English. Fujitsu-published under MIT.
Comparing AI agent performance in multilingual contexts, testing robustness and security against adversarial inputs, and evaluating cross-language generalisation.
Source: innovatiana.com
How this list was put together
Drawn from the datasets in Dataset Finder filed under this topic, then checked by hand against what this page covers. A tag is a filing decision, not a description, so records that carried the tag without being about the topic were removed, and relevant records filed elsewhere were added. The order puts the closest fit first.
Sizes and licences come from each dataset's own documentation, vague wording included. Where we found a re-upload or catalogue card stating a licence that the original release contradicts — a CC0 tag on Flickr photographs, MIT on data whose terms require a signed agreement — the entry gives the original terms and says so. The comparison table abbreviates licences; hover a cell for the full wording.