Fine-tuning Gemma: licence, template and the small variants

Which Gemma, which licence

Gemma's licence depends on the generation. Gemma 4 (April 2026) is released under Apache 2.0, the first Gemma generation to use a standard open-source licence. Earlier generations — Gemma 1, 2 and 3, and the derivatives built on them — ship under Google's Gemma Terms of Use with an accompanying prohibited use policy.

Those terms permit commercial use, which is the main question most people have, but they are their own document with their own conditions: the use restrictions travel with the model into your deployment and with any model you derive from it. Check the licence attached to your exact checkpoint.

The general principle from our other posts applies here with full force: your fine-tune's permissible uses are the intersection of the base model's licence, your training data's licence, and — if the data was machine-generated — the generating model's terms. Three documents, all of which have to permit what you intend.

The template is its own

Gemma's chat format differs from both Llama's and Mistral's, with its own turn markers and structure, and the format has varied across generations of the family. Unsloth's template collection carries several distinct Gemma entries plus ChatML-flavoured variants, which is a reasonable proxy for how much the details differ.

One characteristic worth knowing: the earlier Gemma releases did not have a separate system role, in the way some Mistral releases did not. If your data carries system messages, check what template rendering does with them rather than assuming: depending on the generation the template either raises an error or folds the system text into the first user turn, and the second looks like success unless you read the output.

Everything in our conversation formats post applies: store conversations as a messages list, never store rendered template tokens in your dataset, and let the tokenizer apply the template at training time.

from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("<gemma-checkpoint>")

msgs = [{"role": "system", "content": "SENTINEL"},
        {"role": "user",   "content": "hi"}]
try:
    rendered = tok.apply_chat_template(msgs, tokenize=False)
except Exception as exc:
    print("template rejects a system role:", exc)   # fold it into the user turn
else:
    print("system text survives:", "SENTINEL" in rendered)
    print(repr(rendered))   # surviving is not the same as a separate system turn:
                            # read whether it sits in its own turn or in the user's

The small variants change the economics

This is the genuinely distinctive thing about the family. Alongside the conventional sizes, Gemma includes very small models — down to Gemma 3's 270M, under the Gemma Terms of Use — and those make projects practical that are not practical at 7B.

What a few-hundred-million-parameter model is good for: a single narrow task, learned well, deployed somewhere a large model cannot go. On-device, in a browser, at the edge, as one stage in a pipeline where you need an answer in milliseconds, or at a volume where per-request cost dominates everything.

What it is not good for: general capability. A small model fine-tuned on one task can do that task creditably and will not hold a conversation about anything else. That is a feature if the task is the product, and a disappointment if you expected a small assistant.

The data implication is encouraging: a narrow task on a small model is exactly the LIMA-territory case where a few hundred to a few thousand consistent examples is the right size, and where curation matters far more than volume. Small models are also less forgiving of inconsistent examples than large ones, so the review time is well spent.

Settings

Standard contract — see the TRL dataset contract for the data shapes. Unsloth's memory table gives roughly 3.5 GB for a 3B model at 4-bit QLoRA, 8.5 GB at 14B and 22 GB at 27B, with 16-bit LoRA needing 8, 33 and 64 GB respectively. Gemma 3's 4B and 12B sizes fall between those rows, and the very small variants are well under them.

Rank 16 to 32, alpha equal to rank or double, around 2e-4 for SFT, one to three epochs. For the smallest models consider the lower end of the rank range: there is less model to adapt, and a large adapter on a small base mostly accelerates overfitting.

The family also includes domain-specialised derivatives — code, medical and embedding variants among them. As with any family, starting from a specialised variant beats teaching a general model the specialisation from scratch.

Questions people ask