Phi is the family built on the quality-over-quantity thesis, and its training recipe is the most interesting thing about it from a data perspective — because it predicts both its strengths and its gaps.
The recipe, and why it matters to you
The original phi-1 result is the clearest published demonstration that curated data beats scale. A 1.3B-parameter model, slightly over 50 billion total tokens seen, trained on about 6B tokens of filtered code plus under 1B tokens of generated Python textbooks and under 180M tokens of generated exercises — reaching 50.6% pass@1 on HumanEval and 55.5% on MBPP, in four days on eight A100s.
Two caveats the paper states itself: it releases the model but does not say the training data was released, and it omits the synthetic-generation details for proprietary reasons. So it is a finding rather than a reproducible recipe.
The relevance to your fine-tune is that the family's character follows from the recipe. Trained disproportionately on clean, pedagogical, reasoning-dense material, these models are strong at reasoning and structured problem-solving relative to their size — and comparatively thin on the messy, long-tail, culturally specific knowledge that a web-scale corpus supplies incidentally.
What this means for your training data
If the model's strength is reasoning and its gap is breadth, your data should play to the strength rather than the gap.
Good use of a Phi fine-tune: teaching your output format, your domain's structured conventions, a specific reasoning pattern you want applied consistently, or a narrow extraction or classification task. These align with what the base model is already built for.
Harder use: supplying broad world knowledge. If your application needs the model to know a great deal about an area the base model saw little of, instruction data will not install that — you are looking at continued pretraining, and at a model whose pretraining breadth may make it a poor starting point for that work regardless.
There is also a pleasing consistency here: the family's own thesis is that a small amount of high-quality data beats a large amount of mediocre data. That applies to your fine-tuning set too. A few hundred carefully written, consistent, pedagogically clear examples is exactly the kind of data this family was built on.
Template and settings
Phi has its own chat format, and Unsloth's collection carries separate template entries for the 3, 3.5 and 4 generations — so again, do not assume one generation's handling transfers. Apply the tokenizer's template and inspect the rendered output once.
Unsloth's memory table puts the 14B-class model at about 8.5 GB for 4-bit QLoRA and 33 GB for 16-bit LoRA. The smaller variants are correspondingly lighter, and the family's capability-per-parameter is the reason to consider it when serving cost matters.
Rank 16 to 32, alpha equal to rank or twice it, around 2e-4 for SFT, one to three epochs, loss on assistant turns only. Nothing about this family changes the data contract — see the TRL dataset contract.
A note on synthetic data, since this family is the argument for it
Phi is the strongest published case for synthetic training data, so it is worth being precise about what it does and does not license you to conclude.
What it supports: deliberately written, pedagogically structured material can outperform much larger quantities of scraped material. The generalisable property is pedagogy rather than synthesis — what made the data good was that it was written to teach, which human-written data can also be.
What it does not support: that generated data is safe for teaching facts. You inherit the generating model's errors with its confidence, uniformly across every example. Our synthetic data post covers the failure modes, the most important of which is that they do not appear in your loss curve and may not appear in your evaluation either if the eval set came from the same generator.
Questions people ask
What licence do Phi models use?
Microsoft has released models in this family under permissive terms including MIT for some checkpoints, which is unusually clean. As always, verify on the specific checkpoint rather than assuming it holds across the family.
Is Phi a good base for a general assistant?
Less so than families trained on broader corpora. Its relative strengths are reasoning and structured tasks at small size; general conversational breadth and long-tail knowledge are where it is thinner. For a narrow capable component it is an excellent choice.
Does the synthetic pretraining affect my fine-tuning?
Mainly through what the model already knows. You are fine-tuning a model whose pretraining emphasised clean reasoning-dense text, so it adapts readily to structured tasks and may need more help with informal register or niche factual domains. Test both on your evaluation set.
How much data do I need?
The published figures come from Unsloth's documentation, not from the model's maker: a minimum of 100 rows with over 1,000 preferable for supervised fine-tuning, at 1–3 epochs, and at least 500 rows for GRPO. Nothing is published for DPO-style preference tuning. See how much data for a LoRA for the doubling experiment that answers it for your task rather than in general.