Non-speech audio is a smaller category than speech and covers a wider range of real applications, from predictive maintenance to wildlife monitoring.
The granularity question decides whether a dataset fits your task. Clip-level labels say a sound occurred somewhere in the clip. Frame-level or timestamped labels say when. Only the second lets you measure localisation in time. Clip-level (weak) labels can still train a detector — weakly supervised sound event detection, as in DCASE Task 4, learns onsets and offsets from clip labels — but expect coarser timing, and you still need some timestamped data to evaluate it. Check which you are getting, because both are described as sound event data.
The second issue is label overlap. Real audio contains simultaneous sources, so single-label datasets implicitly assume one dominant sound, which is often false. If your application involves a machine running while someone talks nearby, you need multi-label data with overlapping annotations, and that is a much smaller pool.
| Dataset | Modality | Size | Licence | Best for |
|---|---|---|---|---|
| ESC-50 (Environmental Sound Classification) | Audio | 2,000 clips | CC BY-NC | Supervised audio classification training |
| AudioSet | Audio | 2M clips | CC BY 4.0 (labels only) | Environmental sounds |
| UrbanSound8k | Audio | 8,732 clips | CC BY-NC 4.0 | Urban audio |
| Tropical Gunshot WDA Dataset | Audio | 7,300 clips | MIT | Audio recognition to detect gunshots |
| DCASE Challenge Dataset | Audio | Several thousand audio recordings | Research only | Acoustic scene classification |
| VGGsound: Audio-Visual Dataset for Natural Sounds | Multimodal | 200,000 videos | CC BY 4.0 | Sound recognition on real-world content |
| Respiratory Sound Database | Audio | 1,844 files | Research only | Respiratory pathologies |
ESC-50 (Environmental Sound Classification)
- Size
- 2,000 audio clips of 5 seconds each, WAV format
- Licence
- Free under a Creative Commons Attribution NonCommercial License (CC BY-NC)
2,000 five-second WAV clips of labelled environmental sounds spanning everyday, natural and urban categories. Small and balanced, it is the usual first benchmark for audio classification, though the non-commercial clause limits product use.
Supervised audio classification training, validating methods on real-world sounds, building embedded audio recognition, and benchmarking new algorithms.
Source: innovatiana.com
AudioSet
- Size
- Over 2 million labelled clips (ten-second YouTube segments): CSV label files, ontology in JSON; audio not distributed
- Licence
- CC BY 4.0 for the labels (the ontology is CC BY-SA 4.0); the audio stays on YouTube under its owners' rights
Over two million ten-second YouTube sound clips labelled with 527 sound classes from an ontology of more than 600, with CSV label files from Google and audio extracted from YouTube rather than redistributed. The hosting arrangement matters for reproducibility.
Classifying and detecting environmental sounds, building real-time sound recognition, annotating complex audio scenes for robotics, and analysing sound events.
Source: innovatiana.com
UrbanSound8k
- Size
- 8,732 audio clips, WAV format
- Licence
- CC BY-NC 4.0 on the Zenodo download (the project site cites 3.0): non-commercial use only
8,732 WAV field recordings of city sounds such as sirens, car horns and construction work, organised into ten classes with pre-defined folds. The fold structure makes cross-validation results comparable across papers.
Recognising and classifying urban audio, building noise pollution monitoring, improving ambient sound detection for smart cities, and studying urban soundscapes.
Source: innovatiana.com
Tropical Gunshot WDA Dataset
- Size
- About 7,300 WAV clips (16 kHz mono; 7,000 train, 300 validation; half gunshot, half background), 468 MB
- Licence
- MIT
About 7,300 WAV clips at 16 kHz mono for binary gunshot detection in tropical forest: half gunshots and half background. The 3,500 training gunshots are augmented from the 597 originals of Katsis et al. (2022); the 300-clip validation split holds 150 original gunshots and 150 backgrounds.
Training audio recognition to detect gunshots, developing security and noise monitoring in tropical areas, and improving classification in noisy environments.
Source: innovatiana.com
- id
- /m/05tny_
- name
- Bark
- description
- Principal communication sound produced by dogs. Often transliterated as woof, especially for large dogs.
- positive_examples
- youtu.be/PGfUJCl4M2E?start=80&end=90 · youtu.be/o-a6xZnOvP4?start=40&end=50 · youtu.be/G8BSfIQcDI0?start=110&end=120 … 8 in all
- child_ids
- []
DCASE Challenge Dataset
- Size
- Several thousand audio recordings, WAV format
- Licence
- Free for academic and research use (DCASE specific license)
A pointer to the DCASE challenge rather than one dataset (its source link is the DCASE 2024 challenge page): each task ships its own data, acoustic scenes among them, with its own size and terms. Released as part of the annual DCASE challenge series, so splits and tasks change year to year.
Training acoustic scene classification, developing algorithms that identify acoustic context, benchmarking multimodal approaches, and building acoustic monitoring for smart cities.
Source: innovatiana.com
| YouTube id | start (s) | label | split |
|---|---|---|---|
| --0PQM4-hqg | 30 | waterfall burbling | train |
| -021-GT2EIM | 174 | church bell ringing | train |
| -3-4qmWSJXU | 30 | thunder | test |
VGGsound: Audio-Visual Dataset for Natural Sounds
- Size
- Over 200,000 video/audio clips, 10-second segments, 310 sound classes, CSV files for metadata
- Licence
- Creative Commons Attribution 4.0 International License (CC BY 4.0)
Over 200,000 ten-second audio-visual clips of everyday sounds taken from YouTube videos, spanning 310 sound classes with CSV metadata. The paired audio and video make it a staple for audio-visual correspondence learning.
Training sound recognition on real-world content, developing multimodal audio-video analysis, and evaluating deep learning architectures on in-the-wild data.
Source: innovatiana.com
Respiratory Sound Database
- Size
- 1,844 files: 920 WAV recordings and 920 TXT annotation files, plus diagnosis and demographic metadata files
- Licence
- Free to use for research purposes
920 annotated lung sound recordings with matching text annotation files, from 126 patients, labelled for abnormal breathing sounds such as crackles and wheezes, plus diagnostic and metadata files. Use is limited to research purposes.
Detecting respiratory pathologies such as asthma or COPD, automating medical sound annotation, and building smart stethoscope applications.
Source: innovatiana.com
How this list was put together
Found by searching Dataset Finder for datasets whose documentation describes the task, then checked by hand. A keyword match is not a fit — the first pass picked up a coral survey for the depth-estimation list because it was recorded below fifty metres — so every entry was read against what this page covers, and the ones that matched a word without matching the task were removed.
Sizes and licences come from each dataset's own documentation, vague wording included. Where we found a re-upload or catalogue card stating a licence that the original release contradicts — a CC0 tag on Flickr photographs, MIT on data whose terms require a signed agreement — the entry gives the original terms and says so. The comparison table abbreviates licences; hover a cell for the full wording.