Having read several hundred dataset pages while building this site's dataset posts, we find the pattern consistent: cards describe what the data is and omit what you need to decide whether to use it. Here is what to include.
The four fields that are most often missing
Ranked by how often their absence blocked us from recommending an otherwise good dataset.
The licence, stated unambiguously. Not “open”, not “freely available”, not a Creative Commons logo without a variant. A specific identifier. Of the 434 datasets we verified, several state wording such as “Creative Commons (exact type to be specified)” or “Unknown license (use to be verified)” — which we preserved verbatim rather than tidying, because the vagueness is the information. A dataset with an unclear licence is unusable for commercial work however good it is.
Whether it contains personal data. One line. It determines whether an entire regulatory regime applies, and the person reading your card cannot infer it from a size figure.
What generated it, if anything. Most instruction data is now machine-generated. Naming the teacher model tells the reader what behaviour they are importing and whose terms also apply. The best open datasets do this; many do not.
Known limitations. Every dataset has them, and a card that lists none reads as unexamined rather than as perfect. This is the field that builds the most trust.
The checklist
Organised by the question each field answers.
- What is it? One-sentence summary. Modality. Number of records, stated precisely, with a note if that differs from the name. Size on disk. Languages or domains covered.
- May I use it? Licence identifier. A link to the full text. Any use restrictions beyond the licence. Whether access is gated and how to request it.
- Where did it come from? Sources, named. Collection period. Collection method — scraped, commissioned, generated, aggregated. If generated: the model, version and prompting approach. If aggregated: the component list with each component's licence, since a headline licence rarely covers mixed subsets uniformly.
- What is the structure? Field names with types. An actual example record. The format. Splits and how they were made — random, chronological, by source.
- How was it labelled? Who annotated. The guidelines, or a link. Inter-annotator agreement if measured. Whether labels are human, machine or mixed, and if mixed, how to tell which is which.
- Does it contain personal data? Yes, no, or pseudonymised. Of what kind. What de-identification was applied and to which standard. Whether subjects consented, and to what.
- What is wrong with it? Known errors and their rate if measured. Known gaps in coverage. Class imbalance. Biases identified. Anything you would warn a colleague about.
- How do I cite it? A citation block, and a contact for questions.
Write the limitations section properly
This is the section that distinguishes a card written by someone who used the data from one written to accompany a release.
Useful content: measured label error rate from a sample audit; which classes or groups are thin; what the data does not cover that a reader might assume it does; known artefacts such as watermarks, boilerplate or duplicated records; how much the splits overlap if at all; and the ways you have seen people misuse it.
That last one is particularly valuable and almost never present. If you know that people keep treating your clip-level labels as if they carried timestamps, or using your de-identified data without checking their own jurisdiction's standard, say so.
The incentive here is not purely altruistic. A card with honest limitations gets your dataset used appropriately and cited, rather than used badly and blamed.
If you are reading rather than writing
The inverse checklist, as a triage procedure. These four steps take about ten minutes and eliminate most unsuitable candidates before you download anything.
Check the licence first and stop if it is unclear or restrictive for your use. Check for personal data second, since it determines whether a compliance conversation is needed. Read twenty actual records third — not the summary, the records — and ask whether they resemble your production inputs. Check whether the split methodology would leak, especially for video, audio or time-series data.
Then, before training, verify the counts yourself. Published figures are wrong often enough — and contradict dataset names often enough — that counting the rows you actually downloaded is worth the thirty seconds. Record what you found in your provenance log — our provenance post covers what to keep.
Questions people ask
Is there a standard format for dataset cards?
Several conventions exist, including the datasheet framing from the research literature and the structured metadata the major hubs support. Any of them beats prose, and the structured hub metadata has the advantage of being machine-readable, which is how datasets become findable.
How much detail about annotation is enough?
Enough that a reader could reproduce your labelling decisions on a new example. In practice: the guidelines themselves, how edge cases were resolved, and the agreement rate. Those three let someone judge whether your labels mean what they need them to mean.
Should I document a dataset I only use internally?
Yes, and the internal case is where it pays off most — your future colleagues are the readers, and they will have none of the context you have now. The same checklist applies; you can skip the citation block.
What if I do not know the answer to a field?
Write that you do not know, rather than omitting the field. “Collection method not recorded” is informative and honest; silence reads as either unexamined or evasive, and the reader cannot tell which.