Provenance is the least interesting work in machine learning and the one whose absence causes the most avoidable trouble. It costs minutes per dataset while you are collecting and cannot be recovered once you are not.
The questions it answers
Provenance feels like paperwork until someone asks one of these, at which point it is the difference between an answer and a shrug.
May we ship this model commercially? Requires knowing every dataset's licence. A dataset we used has been withdrawn — are we affected? Requires knowing which model versions used it. Someone requested erasure — was their data in our training set? Requires knowing which sources contained personal data. Our model is biased toward X — why? Requires knowing the composition. Can we reproduce the model from eighteen months ago? Requires knowing the exact data and processing.
Every one of these questions arrives eventually, usually at an inconvenient moment, and every one is answerable in minutes with a record and not at all without one.
What to record
One row per dataset, in version control next to your training code. This is the minimum that answers the questions above.
- name: example-instruction-mix
version: "1.2"
source: https://example.org/datasets/example-mix
obtained: 2026-10-14
licence: "CC BY 4.0"
licence_copy: licences/example-mix-CC-BY-4.0.txt # the text AS IT STOOD
checksum: "sha256:9f2b..." # what you actually have
rows: 48213
personal_data: none # none | pseudonymised | present
generated_by: null # the teacher model, if any
generator_terms: null # and ITS terms
processing:
- deduplicated against eval set (minhash, 0.70)
- filtered rows with empty completions (-312)
used_by: [model-v3, model-v4]
notes: "component list includes two non-commercial subsets - excluded, see script"
The fields people omit and later need
Four in particular, each of which is the answer to a question that arrives later.
A copy of the licence text, dated. Not a link — a copy. Licences change, pages move, and your evidence of good faith is what the terms said when you relied on them. This is the single highest-value field in the record and the most commonly skipped.
A checksum. Datasets get silently updated. A checksum tells you whether what you have is what you had, which is the difference between a reproducible run and a plausible story.
What generated it, and under what terms. Most open instruction data is now machine-generated. Your model's permissible uses depend on the generating model's terms as well as the dataset's licence, and that information lives nowhere except the card you read once. Our guide to dataset licences covers why licences stack.
Which model versions used it. This is the field that turns a withdrawal or an erasure request from a crisis into a lookup. It also makes “why did v4 regress” answerable.
Per-sample provenance, where it matters
Dataset-level records suffice for most work. Per-sample provenance is worth the extra effort in specific cases.
Mixed-source datasets where components carry different licences — common in large instruction mixtures, where a headline licence covers subsets with their own terms. A single dataset-level licence is simply inaccurate there.
Data containing personal information, where you may need to locate and remove specific records.
Anything you may need to audit for bias or quality, where knowing that the problematic examples all came from one source is the finding.
The implementation is a source field on every row, carried through every transformation. It is easy to add at ingestion and extremely tedious to add later, and the formats mostly accommodate it — COCO has a per-image licence field, and for formats without a slot a sidecar manifest keyed by identifier works.
Make it a habit, not a project
Provenance fails when it is a documentation task scheduled for later, because later does not come and the information decays.
What works is making the record a precondition: a dataset is not usable until it has a row in the file. Put the file in version control so changes are reviewed. Have your data-loading code read it, so an undocumented dataset cannot be trained on by accident — that single trick does more than any amount of policy.
And write a dataset card for anything you produce, using the same discipline you want from others. Our dataset card checklist has the fields. The datasets that are pleasant to use are the ones whose authors did this, and the courtesy is reciprocal.
Questions people ask
Is a model card not enough?
A model card is the public summary; provenance is the internal record behind it. The card says which datasets were used, the record says where each came from, under what licence as it stood, with what checksum and what processing. You need the second to write an honest first.
What if I inherited an undocumented pipeline?
Document forward rather than attempting full archaeology. Record everything from today, and reconstruct what you can for datasets still in active use — prioritising licence and personal-data status, since those are the ones with consequences. Partial records beat none.
Does this apply to synthetic data I generated?
Especially. Record the generating model and version, the prompts or generation script, the date, and the generating service's terms. Synthetic data is the case where provenance is most often skipped and most often needed, because the terms question is invisible in the data itself.
How do I track provenance through transformations?
Carry a source identifier on every row from ingestion, and have each transformation preserve it. Log counts in and out at each step, which both preserves lineage and catches the quiet row loss that our deduplication post warns about.