Depth datasets differ most in how the ground truth was obtained, and that provenance determines what a model trained on them can actually do.
There are three sources of depth ground truth and they are not equivalent. Active sensors give metrically accurate depth with characteristic failure modes — holes on reflective, transparent and distant surfaces. Stereo matching gives dense depth that is wrong in exactly the regions where matching is ambiguous. Synthetic rendering gives perfect, complete depth for scenes that do not look quite real.
The practical consequence is that a model trained on one source inherits that source's blind spots. Sensor-depth training data teaches a model to be uncertain about windows because the sensor was. If your application involves glass, water or shiny metal, check what the ground truth did there before anything else.
This list is short because the verified set is. The depth benchmarks most papers report — outdoor driving sets above all — are not among the datasets checked for this page, and the ones that are lean indoor: RGB-D scans of rooms and reconstructions of scenes. That matters more than the count. A model trained on indoor sensor depth has learnt ranges of a few metres and the statistics of furniture, and it will not tell you how far away a car is. If your target is outdoors, treat everything below as a pretraining source and plan to evaluate on data from your own environment, checking the terms of any outdoor benchmark before you build on it — several of the best known are non-commercial.
| Dataset | Modality | Size | Licence | Best for |
|---|---|---|---|---|
| NYU Depth V2 | Multimodal | 47,584 images | Unclear — see below | Monocular depth estimation for indoor scenes |
| ScanNet | Image | 1,513 scans | Gated access | 3D semantic segmentation |
| SceneSplat-7K | Image | 7,916 scenes | Research only | NeRF or 3D Gaussian Splatting models |
| CV-Bench | Multimodal | 2,638 examples | Apache 2.0 | Assessing 2D and 3D abilities |
NYU Depth V2
- Size
- 47,584 training and 654 validation RGB images with dense depth maps (FastDepth preprocessing), Parquet
- Licence
- No licence stated on the official NYU page (citation requested); this Hugging Face copy's card says MIT and its metadata tag Apache 2.0
RGB images paired with dense depth maps from NYU Depth V2's indoor Kinect recordings, in the FastDepth preprocessing: 47,584 for training and 654 for validation. This copy holds depth only; the segmentation labels of the original 1,449-image labelled set are not in it. The long-standing baseline for monocular depth estimation.
Training monocular depth estimation for indoor scenes, building perception for home robots or AR/VR glasses, and studying 3D scene reconstruction from RGB-D.
Source: innovatiana.com
ScanNet
- Size
- 1,513 annotated RGB-D scans, 2D/3D formats (images, voxels, meshes, depth maps)
- Licence
- ScanNet Terms of Use for the data (signed agreement required); code under MIT
1,513 annotated indoor RGB-D scans covering 20 object classes, delivered across 2D and 3D formats including images, voxels, meshes and depth maps. The reference dataset for indoor 3D semantic segmentation. Only the code is MIT: the data is released under the ScanNet Terms of Use, which you sign to request access.
3D semantic segmentation, interior scene reconstruction, object detection and tracking in enclosed spaces, and robotic perception and indoor navigation.
Source: innovatiana.com
SceneSplat-7K
- Size
- 7,916 scenes, 11.27 billion 3DGS points, `.npy`, `.json` files and metadata in `.csv`
- Licence
- CC BY-SA 4.0 covers only the code, scripts and metadata; the scenes come from ScanNet, Matterport3D and other indoor sets, which restrict use to non-commercial research and education
7,916 indoor 3D scenes reconstructed from sources including ScanNet and Matterport3D, carrying 11.27 billion Gaussian Splatting points with poses and annotations in .npy, .json and .csv. Built for 3DGS and NeRF work. Access is gated, and the CC BY-SA 4.0 licence covers only the code and metadata: every source dataset restricts the scenes to non-commercial research and education. The card's statistics after later data updates total 5,384 scenes.
Training NeRF or 3D Gaussian Splatting models for photorealistic rendering, evaluating indoor 3D reconstruction from RGB-D, and testing 3D vision-language architectures.
Source: innovatiana.com
CV-Bench
- Size
- 2,638 examples with 2D/3D annotations and images, 810 MB, Parquet and JSONL
- Licence
- Apache 2.0
2,638 manually inspected examples with 2D and 3D annotations plus associated images, 810 MB shipped as both Parquet and JSONL, drawing labels from ADE20k, COCO and OMNI3D. It tests whether vision-language models grasp spatial layout and depth.
Assessing 2D and 3D abilities of multimodal models, probing comprehension of spatial layout and depth, and benchmarking vision and multimodal research.
Source: innovatiana.com
How this list was put together
Found by searching Dataset Finder for datasets whose documentation describes the task, then checked by hand. A keyword match is not a fit — the first pass picked up a coral survey for the depth-estimation list because it was recorded below fifty metres — so every entry was read against what this page covers, and the ones that matched a word without matching the task were removed.
Sizes and licences come from each dataset's own documentation, vague wording included. Where we found a re-upload or catalogue card stating a licence that the original release contradicts — a CC0 tag on Flickr photographs, MIT on data whose terms require a signed agreement — the entry gives the original terms and says so. The comparison table abbreviates licences; hover a cell for the full wording.