LLaMA-Factory datasets: the dataset_info.json mapping layer

Published 7 October 2026 · 4 min read

LLaMA-Factory's draw is that you can fine-tune without writing code — a CLI and a Gradio web UI over more than a hundred supported models. The thing you do have to get right is registering your dataset, and that happens in one JSON file.

Everything goes through dataset_info.json

LLaMA-Factory does not infer your data's shape. Every dataset is registered in dataset_info.json under a name, and training refers to it by that name. The entry declares where the file is, which format it is in, and — crucially — how your column names map onto the ones the trainer expects.

This is more ceremony than Axolotl's inline YAML, and it buys the most explicit remapping system of any of these tools. The columns block accepts prompt, query, response, history, messages, system, tools, chosen, rejected, kto_tag, images, videos and audios.

The tags block then remaps the values inside a messages list — role_tag, content_tag, user_tag, assistant_tag, observation_tag, function_tag and system_tag. Between the two you can register almost any conversational data without touching the files.

dataset_info.json

{
  "my_alpaca_data": {
    "file_name": "train.json",
    "columns": {
      "prompt": "instruction",
      "query": "input",
      "response": "output",
      "system": "system_prompt"
    }
  },

  "my_chat_data": {
    "file_name": "chats.jsonl",
    "formatting": "sharegpt",
    "columns": { "messages": "dialogue" },
    "tags": {
      "role_tag": "speaker",
      "content_tag": "utterance",
      "user_tag": "customer",
      "assistant_tag": "agent"
    }
  },

  "my_preference_data": {
    "file_name": "prefs.json",
    "ranking": true,
    "columns": {
      "prompt": "instruction",
      "chosen": "good_answer",
      "rejected": "bad_answer"
    }
  }
}

The registry entry names the parts of your file the trainer needs: the list of turns, the keys that hold the speaker and the text, and the values that mean user and assistant. The file itself is never rewritten, so a different file only needs a different entry.

Two formattings: alpaca and sharegpt

The formatting key selects between two families. Omitted, it defaults to the Alpaca style: flat per-row fields, where prompt and the optional query form the input and response is the target. Multi-turn is possible through a history column of exchange pairs.

Set formatting: sharegpt and it expects a list of turn objects instead, with roles named by the tags block. The default role names are human and gpt, with observation and function_call available for tool-use data — which is more structure than most of these tools offer for agents.

Two line charts for Llama-3.1-8B: left plots OpenOrca perplexity (lower is better) and right plots average GLUE score across 8 classification tasks (higher is better), both against number of trainable parameters on a log x-axis, comparing a solid LoRA series with a dashed BD-LoRA (TP=8) series, points annotated with ranks 16 to 512. — Quality against LoRA rank (16 to 256; BD-LoRA dashed) for Llama-3.1-8B on a language-modelling and a classification task. Each doubling of rank helps, but by little: perplexity moves from about 2.32 to 2.29 and the GLUE average by about half a point, so a rank picked anywhere sensible matters far less than whether the examples underneath it are consistent. Image: huggingface/peft · Apache-2.0 repository

Preference and KTO data

Preference tuning needs "ranking": true on the dataset entry plus chosen and rejected columns. Without the ranking flag a DPO or reward-model run stops with a terse error — "The dataset is not applicable in the current training stage" — that never mentions the flag.

A grouped bar chart titled 'Does the aligned model beat the SFT target?', of win rate minus 0.5 (−0.5 to 0.2) for offline PPO, SFT+offline PPO, DPO, SFT+DPO, KTO and SFT+KTO. Red Pythia bars are all strongly negative; blue Llama bars are small, with the largest positive values for KTO, SFT+DPO and SFT+KTO. — How often aligned Pythia (red) and Llama (blue) models of several sizes beat the supervised fine-tuning target, for offline PPO, DPO and KTO, with and without SFT first. On the Llama models KTO alone does about as well as SFT followed by DPO. KTO needs only a good-or-bad label per answer, the simpler data format LLaMA-Factory accepts alongside preference pairs. Image: Figure 4 of the paper · CC BY-SA 4.0 · from Ethayarajh et al. 2024, “KTO: Model Alignment as Prospect Theoretic Optimization”, arXiv:2402.01306

KTO data takes a different shape: a single completion per prompt plus a boolean in a column mapped to kto_tag. This is the cheaper annotation route — a thumbs up or down on one answer rather than a comparison between two — and worth considering if you are collecting feedback from real users, who will rate an answer far more readily than they will compare a pair.

Two histograms under the title 'Token Length Distribution', one for the train split and one for the test split, each plotting the token lengths of input_ids_prompt, input_ids_chosen and input_ids_rejected. Prompt lengths spread from about 30 to 85 tokens; the chosen and rejected distributions sit further right, between about 50 and 92, and overlap almost exactly. — Token lengths for the prompt, chosen and rejected columns of a small preference set after loading. Chosen and rejected overlap almost exactly, so length is not what separates them, and both sit to the right of the prompt, consistent with each completion being stored with its prompt. One plot like this after registering a dataset shows whether the columns were read as you meant. Image: allenai/open-instruct · Apache-2.0 repository

Who it is for

The web UI is the real differentiator. If the people choosing hyperparameters are not the people who would write a training script, LLaMA-Factory lowers that barrier further than anything else in this category, and the breadth of supported model families is unusually wide.

Two practical notes. Check that the model you want is supported in the version you install rather than assuming the README's count covers it. And because configuration lives in a JSON registry rather than next to the run, keep that file in version control — it is the only record of how your data was interpreted.

Questions people ask

Can I load a dataset from the Hub instead of a local file?

Yes — use hf_hub_url in place of file_name in the registry entry. The column and tag mappings work identically, which is useful when a public dataset uses field names that do not match the defaults.

What happens if my column names already match the defaults?

You can omit the columns block entirely. An Alpaca-style file with instruction, input and output needs only file_name. The mapping layer exists for data you do not control.

Is the data portable to other frameworks?

The underlying files are, since they are ordinary JSON or JSONL — and because the mapping lives in the registry rather than in the data, nothing LLaMA-Factory-specific is baked in. The registry entry itself does not transfer; you would write the equivalent mapping in Axolotl's YAML or a formatting function for TRL.

Does it support multimodal data?

Yes — images, videos and audios columns are part of the mapping vocabulary, for the vision and audio language models among the supported families. Check the documentation for the exact per-row shape, which differs by model.

Keep reading

Fine-tuningtorchtune has wound down: what to use instead Fine-tuningFine-tuning with Unsloth: what your dataset has to look like Fine-tuningFine-tuning Llama: dataset format and the licence to read first