Open X-Embodiment is the ImageNet moment of robot learning: a pooled archive large and diverse enough that pretraining on it transfers. It is also the dataset whose licensing most people get wrong.
What it is
Open X-Embodiment pools existing robot datasets into one format with a coarsely aligned 7-D action space. The published scale: “1M+ real robot trajectories spanning 22 robot embodiments”, assembled from 60 existing datasets contributed by 34 robotics research labs through a collaboration of 21 institutions, covering 527 skills across 160,266 tasks. Those are the paper's figures; the archive has kept growing, and by OpenVLA's count in mid-2024 it held more than 70 datasets and over two million trajectories.
The unification is the valuable part, and it is looser than it sounds. Actions take the shape of “a 7-dimensional action vector controlling the end-effector (x, y, z, roll, pitch, yaw, and gripper opening or the rates of these quantities)”, with each dataset's actions converted to that end-effector shape but not to a common convention: coordinate frames are not aligned, values may be absolute or relative positions or velocities, and each dataset is normalised separately, so the same vector can mean different motions on different robots. The shared shape is what lets one model train across 22 embodiments.
It ships in RLDS, “which saves data in serialized tfrecord files” — not LeRobot format, which is why community LeRobot ports exist and are widely used.
gsutil -m cp -r \
gs://gdm-robotics-open-x-embodiment/{dataset_name} \
~/tensorflow_datasets/
# the RT-1-X JAX checkpoint lives under the same bucket.
# Pick constituent datasets deliberately - OpenVLA, for instance, filtered to
# manipulation datasets with at least one third-person camera.
The licence catch
The official repository states that “all software is licensed under the Apache License, Version 2.0” and “all other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY)”.
Read carefully, that covers the aggregate packaging. The repo states no per-dataset licences — it asks that contributors' datasets be cited separately, and the per-dataset list lives in an external spreadsheet.
So “Open X-Embodiment is CC-BY” is true of the collection as assembled and not necessarily of every one of the 60 constituent datasets, which came from 34 different labs under whatever terms each chose. If you are pretraining a commercial model, that distinction is the whole question, and resolving it means tracing the constituent datasets you actually train on.
It is tedious and it is much cheaper than discovering the problem after training.
What it demonstrated
The paper's own results establish that cross-embodiment pretraining works. RT-1-X's mean success rate is “50% higher” than the original methods on small-scale datasets, beating them on four of five. RT-2-X roughly triples RT-2's success on emergent skills, a gain the paper's conclusion calls “~3× generalization improvements” over a same-embodiment-only model.
The emergent-skills figure is the striking one: success rose from 27.3% for RT-2 to 75.8% for RT-2-X at 55B, while general generalisation stayed roughly level at 62% versus 61%. On Bridge at Stanford's IRIS lab: original method 13%, RT-1 40%, RT-1-X 27%, RT-2-X 50%. Evaluation totalled 3,600 trials across six robots.
Note that RT-1-X underperforms RT-1 on Bridge and on the Google Robot (73% against 92%) while outperforming on the small-scale datasets — pooled pretraining is not uniformly better on every platform, which is a useful corrective to the headline.
OpenVLA is the clearest downstream datapoint
OpenVLA is the most useful reference if you are planning your own training run, because it publishes its costs.
It trained on “a diverse collection of 970k real-world robot demonstrations” drawn from a pool of “more than 70 individual robot datasets”, filtered to “only manipulation datasets with at least one 3rd person camera”.
The result: a 7B open model that outperforms “closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks” with “7x fewer parameters”. The cost: “trained on a cluster of 64 A100 GPUs for 14 days, or a total of 21,500 A100-hours” at batch size 2048.
That is the number to plan against. Cross-embodiment pretraining from scratch is a five-figure GPU-hour commitment; fine-tuning an already-pretrained VLA on your own demonstrations is not, which is why most teams do the latter.
How to use it
The standard recipe is pretrain on pooled cross-embodiment data, then fine-tune on in-domain demonstrations from your own robot. In practice most teams skip the first step by starting from a released checkpoint — OpenVLA, the Pi0 family, GR00T-derived policies — all of which were pretrained on mixes drawn from this archive.
Then collect your own top tier. No amount of pooled data contains your robot in your environment, and the published scaling work is clear that environment and object diversity in that in-domain set matters more than its raw size.
If you do pretrain, select constituent datasets deliberately rather than taking everything: it reduces compute, improves the mix's relevance to your embodiment, and makes the licence audit tractable.
Questions people ask
Do I need the whole dataset?
Almost certainly not. It is a pooled archive of 60 datasets and you select from it — by embodiment similarity to your robot, by task relevance, by camera configuration. OpenVLA's filter to third-person-camera manipulation datasets is a good model, and selection also makes the licensing question answerable.
What is RLDS?
Reinforcement Learning Datasets, a convention for storing episodic step data, serialised into TFRecord files and read through TensorFlow Datasets. It predates the LeRobot format and is why a conversion step sits between OXE and much of the current PyTorch tooling.
Does the 7-dimensional action space limit what I can learn?
It is a lowest-common-denominator representation, so yes — it suits end-effector control of a single arm with a simple gripper and does not natively express dexterous hands, bimanual coordination or whole-body motion. Datasets with richer action spaces exist precisely because of this, and newer bimanual humanoid corpora do not fit the convention.
Is it still the standard, or has something replaced it?
It remains the default cross-embodiment pretraining mix for open VLAs. Larger single-source corpora now exist — AgiBot World is longer by recorded hours (about 2,976 h from 1M+ trajectories) — but are non-commercial and single-platform, so they complement rather than replace a permissively packaged multi-embodiment pool.