Manipulation gets most of the attention, but embodied AI also covers agents that navigate, observe and assist. The data for that is mostly human video and simulated environments, with its own distinct licence problems.
Egocentric human video is the base layer
The cheapest way to show a model what tasks look like is footage of people doing them from their own viewpoint. Three corpora dominate, and their licences differ sharply.
Ego4D is unscripted daily life: “3,670 hours of daily-life activity video” from 931 unique camera wearers in 9 countries and 74 locations, with dense timestamped narrations — 3.85M sentences, 1,772 unique verbs, 4,336 unique nouns, and over 250,000 hours of annotator effort. Its licence is a bespoke Ego4D agreement requiring a signed data-use agreement; the paper does not state its terms, and the published document is marked a draft only in its filename. Do not assume commercial use is permitted — that needs a lawyer reading the current agreement.
Ego-Exo4D adds synchronised third-person views of skilled activity: “1,286 hours of video combined”, 740 participants from 13 cities, 123 scene contexts, with three paired language corpora including expert commentary. The paper states no dataset licence.
EgoDex is the 2025 scale jump and the most directly useful for manipulation: Apple Vision Pro footage with 3D hand and finger tracking captured at recording time, “829 hours of 30 FPS video and paired skeletal data with a total of 90 million frames” and “338000 task demonstrations across 194 tabletop manipulation tasks”. Its licence is CC BY-NC-ND — non-commercial and no-derivatives.
| Corpus | Scale | What is paired with the video | Licence |
|---|---|---|---|
| Ego4D | 3,670 h, 931 wearers | dense narrations, 5 benchmark families | bespoke agreement; signature required; published file named as a draft |
| Ego-Exo4D | 1,286 h ego+exo, 740 participants | narrations, play-by-play, expert commentary | signed licence required; terms not published |
| EgoDex | 829 h, 90M frames, 338k demos | 3D hand and finger tracking at 30 Hz | CC BY-NC-ND |
| EPIC-KITCHENS | 31.7 h in GR00T N1's mix | kitchen activity, action labels | see dataset terms |
How much of it models actually use
GR00T N1 publishes its human-video layer in full, which gives a rare view of how these corpora get weighted in practice. Total human video: 181.3M frames, 2,517.0 hours.
The breakdown: Ego4D 2,144.7 hours, HoloAssist 169.6, Ego-Exo4D 123.0, EPIC-KITCHENS 31.7, Assembly-101 19.3, RH20T-Human 16.3, HOI4D 12.4.
Ego4D supplies 85% of it. Which makes its licence situation the central practical question for anyone replicating this kind of mix commercially. The paper does not answer it, and the published draft agreement lists commercial product development among permitted uses but needs a lawyer's reading.
EgoDex’s own comparison table is useful for sizing: Ego4D's human-object interaction subset gives 89k trajectories and 21M frames; DROID gives 76k trajectories, 86 tasks and 19M frames; EgoDex gives 338k trajectories, 194 tasks and 90M frames. EgoDex is substantially the largest by trajectory count — and the most restrictively licensed.
What egocentric video can and cannot teach
It is worth being precise, because the pyramid framing can imply more than the data delivers.
What it provides: visual representations of objects, scenes and activities; the structure and ordering of tasks; what hands do, and increasingly where they are in three dimensions; language grounded in activity through the narrations.
What it does not provide: robot actions. There is no joint state, no gripper command, no action vector. A model trained on it alone knows what making coffee looks like and cannot actuate anything.
That is exactly why the pyramid exists: human video for representation and semantics, cross-embodiment robot data for action grounding, in-domain teleoperation for the specific platform. The hand-tracking corpora narrow the gap by supplying 3D hand pose, which is closer to an action than pixels are — and the non-commercial licensing on the largest of them is the obstacle to using that.
Where to start
For representation learning from video, V-JEPA 2 is the most encouraging published result — pretrained on over a million hours of internet video, then action-conditioned on under 62 hours of existing robot footage, controlling real arms in labs where no data was collected. Weights are released, so this is a realistic starting point rather than an industrial-scale undertaking.
For anything commercial, resolve licences before you train. The pattern in this field is that the largest and most attractive corpora carry the most restrictive terms, and that several primary sources simply do not state terms at all. An unstated licence is not a permissive one.
And expect to supplement with your own data regardless. Egocentric corpora are recorded in other people's kitchens and workshops; the environment you are deploying into is not in them.
Questions people ask
Can I use Ego4D commercially?
We cannot tell you, and anyone who tells you confidently from the paper has not read it. The paper says only that the data is available under an Ego4D licence without stating terms, the agreement requires a signature, and the published document's filename marks it a draft. This is a question for a lawyer reading the current agreement.
Is egocentric video enough to train a robot policy?
No, on its own. It contains no actions. It is the base layer that supplies representations and task semantics, and it needs robot data above it for action grounding. The hand-tracking datasets get closer by providing 3D pose, but a pose is still not a joint command.
What does CC BY-NC-ND actually prevent?
Non-commercial restricts use to non-commercial purposes; no-derivatives restricts distributing modified versions. Whether training a model constitutes creating a derivative work is legally unsettled, so the combination is the most restrictive you will meet on a major dataset and the one to raise with counsel earliest.
Are there permissively licensed egocentric alternatives?
Smaller ones exist, and the component list in GR00T N1's mix is a reasonable place to start looking — but check each individually, since several of the sources in this area state no licence at all. The large permissive option in embodied data generally remains DROID, which is robot rather than human video.