These eight are less a shortlist than the furniture of computer vision. Most tooling assumes at least one of them, and two of the formats have outlived the datasets that introduced them.
COCO JSON and VOC XML are now interchange formats in their own right. You will read and write them long after you stop training on the datasets.
The licensing is stricter than people assume: ImageNet is non-commercial research, CelebA and Pascal VOC are academic, and ScanNet's data needs a signed terms-of-use agreement (only its code is MIT). Even COCO is CC BY 4.0 for the annotations only; the images remain under their Flickr owners' terms.
| Dataset | Modality | Size | Licence | Best for |
|---|---|---|---|---|
| COCO | Image | ~330,000 images | Mixed — see below | Detection + segmentation |
| ImageNet | Image | 1.28M images | Research only | Backbone pre-training |
| Open Images | Image | 9M+ images | CC BY 4.0 | Scale detection |
| Pascal VOC | Image | 20,000 images | Research only | Legacy baseline |
| CelebA | Image | 200,000+ face images | Research only | Facial attributes |
| Labeled Faces in the Wild | Image | 13,000+ images | Not stated | Verification |
| ScanNet | 3D | 1,513 RGB-D scans | Gated access | Indoor 3D |
| NYU Depth V2 | Image | 1,449 images | Not stated | Depth estimation |
COCO
- Size
- ~330,000 images
- Licence
- Annotations CC BY 4.0; images remain their Flickr owners' copyright under the Flickr Terms of Use
Around 330,000 images with JSON annotations covering detection, segmentation, keypoints and captions. The default benchmark for object detection, and the format most tooling expects — COCO JSON has become a lingua franca well beyond the dataset itself.
Object detection, instance segmentation and keypoints. If you report mAP, this is the dataset behind it.
Source: innovatiana.com/datasets
ImageNet
- Size
- 1.28M train images · 1,000 classes
- Licence
- Non-commercial research
1,281,167 training images across 1,000 classes, with 50,000 validation and 100,000 test. Almost every vision backbone you will use was pre-trained here, which is why its class biases quietly propagate into models that never touch it directly.
Classification benchmarking and backbone pre-training. Non-commercial research use only.
Source: innovatiana.com/datasets
Open Images
- Size
- 9M+ images
- Licence
- Annotations CC BY 4.0; images listed as CC BY 2.0 (Google does not verify each image)
Over nine million images with detailed CSV annotations spanning detection, segmentation and relationships. Larger and more permissively licensed than ImageNet, and the relationship annotations support tasks the others cannot.
Large-scale detection, and visual relationship prediction between objects.
Source: innovatiana.com/datasets
Pascal VOC
- Size
- ~20,000 images · 20 classes
- Licence
- Academic research
Around 20,000 images with XML annotations across 20 object categories. Superseded by COCO in scale, but VOC XML remains a format you will meet constantly, and the smaller size makes it a fast iteration loop.
Quick detection experiments, and anything that has to read or write VOC XML.
Source: innovatiana.com/datasets
CelebA
- Size
- 200,000+ face images
- Licence
- Academic · conditions apply
Over 200,000 celebrity face images with attribute annotations in TXT. Widely used for attribute prediction and as a generative benchmark — and widely criticised for demographic skew, which is worth measuring before you build on it.
Facial attribute prediction and face generation. Academic use under specific conditions.
Source: innovatiana.com/datasets
Labeled Faces in the Wild
- Size
- 13,000+ images
- Licence
- No licence stated by UMass; this Kaggle copy is labelled LGPL-3.0
Over 13,000 JPEGs at 250×250, commonly resized to 62×47. The original face verification benchmark: given two photographs, same person or not. Small and long since saturated, but still the quickest sanity check for a verification pipeline.
Face verification baselines and pipeline smoke tests.
Source: innovatiana.com/datasets
ScanNet
- Size
- 1,513 RGB-D scans
- Licence
- ScanNet Terms of Use for the data (signed agreement, access on request); code under MIT
1,513 annotated RGB-D scans of indoor scenes, with 2D and 3D representations including voxels, meshes and depth maps. The multiple representations let you compare approaches on identical scenes. Note the licence split: the code is MIT, but the data is released under the ScanNet Terms of Use, which you must sign before downloading.
3D semantic segmentation, scene reconstruction and indoor robotics.
Source: innovatiana.com/datasets
NYU Depth V2
- Size
- 1,449 annotated images · 407,024 raw frames
- Licence
- Not stated (citation requested)
1,449 densely annotated RGB-D images plus 407,024 raw frames. NYU states no licence and asks only for a citation; the Apache 2.0 tag comes from a Hugging Face Parquet copy that holds a FastDepth-preprocessed subset, not the full release. The reference dataset for monocular depth estimation: train on the raw frames, evaluate on the annotated subset.
Monocular depth estimation and indoor scene understanding.
Source: innovatiana.com/datasets
How this list was put together
All eight are listed in Innovatiana's open-dataset catalogue; each entry links to the page carrying the full licence text and size string.
Image counts are quoted as published. Where train, validation and test splits are given separately, the table shows the training figure.