The published research results range from 1,000 examples to nearly a million, and both ends are defensible. Rather than pick a number from that range, it is faster to run the one experiment that tells you where your own task sits.
The published figures, separated from the folklore
There is remarkably little vendor-published guidance, so it is worth laying out exactly what exists.
Unsloth's documentation recommends “a bare minimum of at least 100 rows” with “over 1,000 rows” preferable, and 1–3 epochs. Among open-source training libraries it is the only row-count recommendation. Hosted fine-tuning APIs publish their own — OpenAI's guide requires at least 10 examples and recommends starting with 50 well-crafted ones, though OpenAI is now phasing fine-tuning out: closed to new users, with existing customers able to start training jobs until 6 January 2027.
From the research side: LIMA fine-tuned a 65B base model on exactly 1,000 curated examples and was judged equivalent or preferable to GPT-4 in 43% of cases. Tülu 3’s SFT mixture is 939,344 samples, drawn from the 23.3M prompts it collected for all its training stages. InstructGPT used about 13k SFT prompts.
For reward-based RL such as GRPO, Unsloth's RL guide suggests at least 500 rows; for preference tuning (DPO, ORPO, KTO), no major framework publishes a row count at all. Any specific number you are offered there is experience, not specification.
Why the range spans three orders of magnitude
The disagreement is not confusion; the two camps are measuring different things.
LIMA’s premise is that instruction tuning selects among behaviours the base model already has rather than installing new capability — so a small, consistent, diverse set is enough to pick the behaviour. Its ablations support this: doubling one source did not improve quality, while diversity of source did, and filtering mattered.
Tülu 3’s premise is coverage. A general assistant must handle maths, code, safety, instruction-following, multilingual chat and tool use to a uniform standard, and each capability bucket needs its own examples. Nine hundred thousand is what covering all of them takes. Tellingly, the paper says they stopped there because they allocated other prompts for preference optimisation, not because more would not have helped.
So: one behaviour, hundreds to a few thousand. Many behaviours, a structured mixture. The question is how many distinct things you need the model to do.
| What you are teaching | Realistic range | Where effort goes |
|---|---|---|
| One output format or schema | 102–103 | consistency — no contradictory examples |
| One domain task done well | 103 | diversity of inputs, careful review |
| Several related tasks | 103–104 | deliberate per-task balance |
| A general assistant | 105–106 | mixture composition and deduplication |
| Domain knowledge the model lacks | 1010+ tokens | this is continued pretraining, not SFT |
The doubling experiment
Rather than guessing a target, measure the slope. This takes an afternoon and answers the question for your task rather than someone else's.
Build an evaluation set first — a few hundred examples, checked by hand, held out and never trained on. Shuffle before splitting: Unsloth's troubleshooting notes warn that an unshuffled selection gives you a wrong evaluation, which is easy to hit when your data arrived grouped by source.
Then train on a quarter of your data, half, and all of it, and plot the evaluation metric against dataset size. The shape of that curve tells you what to do next, and it is a much more reliable guide than any published figure.
- Still climbing steeply at full size — data is your bottleneck. Collecting more is the highest-value action available.
- Flattening — you are near the useful limit of this data. More of the same will not help; better or more varied data might.
- Flat from the start — something else is wrong. Check the masking, the chat template, and whether the examples actually demonstrate the task.
- Rising then falling — the added data is lower quality than the original. Look at what you added most recently.
What the loss curve tells you about size
The loss curve gives you a second, cheaper signal while a run is in progress.
Unsloth's LoRA guidance states that a training loss below 0.2 means the model is likely overfitting. As a rule of thumb, a loss that settles somewhere around 0.5 to 1.0 is usually healthy, though where it should settle depends on the dataset and task.
A loss that plunges toward zero in the first epoch on a small dataset is memorisation: the model has learned your specific examples rather than the behaviour they demonstrate. More epochs will not help; more varied data will. A loss that reads exactly 0.0 from the first steps is also worth checking for a masking bug — if every label is masked there is nothing to learn from — before concluding that the model has overfit.
Quality beats quantity, with a mechanism
The reason curation wins so decisively at this stage is that every example is a demonstration. A thousand examples that all show the behaviour you want give a clean signal. A hundred thousand scraped ones containing contradictory styles, truncated outputs and occasional refusals teach the model to sometimes do each of those — and a good example does not cancel a bad one.
LIMA’s filtered-versus-unfiltered comparison on the same source gives a measurable difference from curation alone. Tülu 3’s SFT mix, about 4% of the 23.3M prompts it collected for every stage, is not a measure of filtering: the same 23.3M pool also supplied its later training stages.
The practical implication for a budget: spend it on review rather than volume. Having a person read and fix a thousand examples usually beats collecting ten thousand more.
Questions people ask
Can I fine-tune on 50 examples?
You can run it. It is below Unsloth's documented 100-row minimum, though OpenAI's fine-tuning guide suggests 50 well-crafted examples as a starting point. For a very narrow format task 50 highly consistent examples can shift behaviour noticeably. Treat it as a pilot that tells you whether the approach is promising, not as a result.
Does model size change how much data I need?
Larger models generally need less to pick up a behaviour, since they have more latent capability to select from — LIMA's result was on a 65B base. Smaller models benefit more from additional examples and are less forgiving of inconsistency.
How much data for continued pretraining instead?
A different regime entirely: 1010–1011 tokens, because you are teaching knowledge rather than behaviour. If you are considering continued pretraining with a few thousand examples, the stage is wrong for the data you have.
Should I add public instruction data to my own?
Usually yes, if licensing permits — it maintains general instruction-following while your data teaches the specific task. Mix deliberately with recorded proportions, deduplicate across sources, and check that your own data is not drowned out by whichever public set is largest.