Synthetic data has gone from a workaround to a mainstream ingredient at every training stage, including pretraining. The results are real and so are the limits, and the limits are mostly invisible until evaluation.
It is no longer a niche technique
The clearest sign of how far this has come: Nemotron-CC includes 1.9 trillion synthetically derived tokens out of 6.3 trillion — not for fine-tuning but for pretraining. Low-quality web documents were rewritten Wikipedia-style by an open 12B instruct model, and high-quality ones were expanded through four further prompts producing diverse question-answer pairs, distillations, knowledge extractions and knowledge lists.
At the post-training end, the large open SFT mixtures are predominantly generated, with named teachers and counts in the millions. Llama 3 reports over 2.7M synthetic examples for code SFT alone.
So the question is no longer whether synthetic data works. It is which jobs it does well.
Where it works well
Three categories, with published support.
- Teaching format and structure. If you need valid JSON against a schema, a consistent citation style, a fixed report layout — generation is ideal. The structure is exactly what a model can reliably produce, and it is checkable, so you can verify every example automatically.
- Expanding a small real seed. Unsloth's guidance suggests having at least ten real examples before generating from them. The real set anchors the distribution; generation supplies volume and variation.
- Curriculum-style content. phi-1 is the striking result here: 1.3B parameters, slightly over 50B total tokens seen, trained on filtered code plus under 1B tokens of generated Python textbooks and around 180M tokens of generated exercises, reaching 50.6% pass@1 on HumanEval and 55.5% on MBPP — in four days on eight A100s plus seven hours. Deliberately written pedagogical content outperformed much larger quantities of scraped code.
Where it fails, and why you will not notice
The failure modes share a property: they do not show up in the training loss, and they often do not show up in evaluation either if your eval set was generated the same way.
Facts the generator does not reliably know. You inherit its errors with its confidence, uniformly across however many examples you generate. This is the big one, and it is why synthetic data is much safer for teaching form than for teaching content.
Narrowed diversity. Generated data clusters. Ask a model for a thousand customer complaints and you get recognisably fewer distinct situations, phrasings and edge cases than a thousand real ones. The model trained on them is correspondingly brittle on the long tail — and the long tail is usually where production failures live.
Contaminated evaluation. If training and evaluation data come from the same generator, your benchmark measures agreement with that generator. The number can look excellent while the model is unusable.
Laundered licence problems. Generating from a model whose terms forbid training competitors does not clear the restriction, and the resulting dataset's own permissive licence does not either.
Verification is what separates the two outcomes
The single highest-value practice is filtering generated data by whether it is actually correct, wherever correctness is checkable. Rejection sampling is the general pattern: Llama 3 generates K candidates per prompt, typically 10 to 30, and keeps only the best of them. When the test that decides which candidates survive is a correctness check rather than a judgement of quality, that is verification in the strict sense.
Maths has an extractable answer. Code has tests. Structured output has a schema. Extraction has a record you hold. In all of these you can generate at volume and keep only what verifies, which converts a quantity advantage into a quality one.
Where you cannot verify — open-ended explanation, tone, judgement — generated data should be sampled and read by a person, and treated as a supplement to real data rather than a replacement. Unsloth's guidance makes the same point plainly: check the generated data and remove or improve poor-quality responses.
A workable division of labour
Putting the above together into something you can plan around:
Use real data for your evaluation set, always, and for anything teaching facts or covering the long tail of situations your users actually produce. A few hundred real, hand-checked examples is the foundation everything else is measured against.
Use synthetic data for volume, for format and structure, for augmenting a real seed, and for domains where verification is programmatic.
And record which is which. A dataset that mixes the two without marking them cannot be audited later, and the first question anyone asks when a model behaves oddly is where the data came from.
Questions people ask
Will training on synthetic data cause model collapse?
The collapse results concern repeatedly training on unfiltered outputs of models trained on outputs, over generations. That is not what the practices here describe — current recipes anchor on real data, filter by verification, and use generation as one ingredient among several. The risk is real in principle and is managed by keeping real data and verification in the loop.
How much of my mixture can be synthetic?
No published threshold, and the answer depends on the job. Nemotron-CC runs roughly 30% synthetic at pretraining scale; several open SFT mixtures are almost entirely generated. The more useful constraint is that your evaluation set should be real, and the facts you depend on should come from somewhere verifiable.
Which model should I generate with?
The strongest one whose terms permit your use — and check that second clause first. Open-weight models with permissive licences are the safe default, and are increasingly what the major open datasets use, though older staples such as UltraFeedback were rated by GPT-4. Also prefer a model that is actually good at your domain; generating clinical text with a general model produces plausible-sounding text with clinical errors.
Can I generate preference pairs?
Yes, and UltraFeedback is essentially that — four responses per prompt from a pool of 17 models, rated by GPT-4 on four aspects. The thing to avoid is rejected answers that are bad in a uniform way, which teaches the model to detect an artefact rather than to prefer quality.