FAQ

NLP

Text classification datasets

Topic, intent, toxicity and sentiment labels — and how to tell whether a public set matches your taxonomy.

Text classification is the workhorse of applied NLP, and the main obstacle to using public data is that your label taxonomy is yours alone.

Almost every production classification problem has a bespoke label set: your support ticket categories, your document types, your routing rules. Public datasets have their own, and mapping between them loses precision at every step — which is why a few thousand examples from your own data usually beats a much larger public set for a production classifier.

Where public data genuinely helps is in establishing a method and a baseline, in pretraining a representation before fine-tuning on your labels, and in the cases where your taxonomy really is standard. Check the label definitions rather than the label names: two datasets can both have a 'complaint' class meaning quite different things.

A line chart of average performance plus or minus variance against the number of labelled training samples on a log axis from 10 to 10,000: specialised fine-tuning (blue) rises steeply and crosses general zero-shot prompting (flat red) at about 30 and few-shot in-context learning (green) at about 70, with dashed lines marking first and second break-even points.
Averaged across the paper's classification tasks: a small fine-tuned model against a large model used zero-shot or with a few examples in the prompt, as labelled training samples grow from 10 to 10,000. Fine-tuning overtakes zero-shot prompting at a few dozen labels and in-context learning at around seventy, more once score variance is counted. Image: Figure 1 of the paper · CC BY 4.0 · from “Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even Performance”, arXiv:2402.12819
DatasetModalitySizeLicenceBest for
E-commerce Text ClassificationText50,425 entriesCC BY 4.0NLP models to classify products
Dataset Banking77 - Banking Customer Service IntentionsText13,083 examplesCC BY 4.0NLP intent detection in banking
SMS Spam CollectionText5,574 messagesCC BY 4.0Text classification for spam filtering
PhilPapers Papers Summarized LabeledText3,776 documentsMITMulti-label classification on academic texts
Civil Comments - Corpus of moderated comments annotated for toxicityText2M commentsCC0Toxicity detection and automated moderation
Chinese Sentiment AnalyzeText182,762 examplesMITSentiment classification in Mandarin
Women's E-Commerce Clothing ReviewsText23,486 rowsCC0Sentiment analysis or review classification
Twitter Sentiment Analysis DatasetText1.6M tweetsTwitter API termsSentiment classification on short texts
Sentiment Analysis for Mental HealthText51,000 textsODbLClassification of mental states from text
Amazon PolarityText4M reviewsApache 2.0Sentiment classification for e-commerce
Licences in this shortlist — computed from the 10 datasets below. Red marks licences that are unclear or restricted; amber marks non-commercial, share-alike or mixed terms.
A line chart titled Polish: Krippendorff's Alpha (0.30 to 0.60) against training-set size steps 1 to 20. A red line with error bars rises slowly from about 0.50 to a peak near 0.54 at step 15 and flattens; a flat blue line marks 0.57.
A Polish sentiment classifier trained on ever larger slices of its labelled set (horizontal axis, in tens of thousands of tweets). Its agreement with the labels (red) peaks at an Alpha of 0.536 after 150,000 tweets, just under the 0.57 the annotators reach with each other (blue). Past that, more labels of the same quality add nothing. Image: Fig 3 of the article · CC BY 4.0 · from Mozetič et al. 2016, PLOS ONE, “Multilingual Twitter Sentiment Classification: The Role of Human Annotators”, doi:10.1371/journal.pone.0155036
№ 01

E-commerce Text Classification

Text
Size
50,425 text entries in CSV, 4 classes
Licence
Attribution 4.0 International (CC BY 4.0)

50,425 product text entries in CSV, each labelled with one of four e-commerce categories. Simple and well-sized for supervised text classification baselines or catalogue auto-tagging prototypes; check the per-class counts before relying on a single accuracy figure.

What to train on it

Training NLP models to classify products from descriptions, setting up automatic categorisation in an e-commerce platform, and testing supervised classification algorithms.

Source: innovatiana.com

№ 02

Dataset Banking77 - Banking Customer Service Intentions

Text
Size
13,083 text examples with intent labels (77 classes)
Licence
CC-BY 4.0

13,083 real banking customer service requests, each labelled with one of 77 fine-grained intent categories. The high class count over short utterances makes it the standard hard intent-classification benchmark.

What to train on it

Training NLP intent detection in banking, improving customer-service chatbots or voice assistants, and analysing user needs and frustrations in detail.

Source: innovatiana.com

№ 03

SMS Spam Collection

Text
Size
5,574 SMS messages, plain text format (TXT/CSV)
Licence
CC BY 4.0

About 5,574 text messages in TXT and CSV labelled spam or ham, gathered from several sources for spam filtering research. Hosted at the UCI repository and small enough to run experiments on in seconds.

What to train on it

Training text classification for spam filtering, NLP research applied to SMS, and evaluating clustering and text analysis techniques.

Source: innovatiana.com

A simple block diagram: Sentence A and Sentence B each pass up through their own BERT box and pooling box producing vectors u and v, which feed a 'cosine-sim(u, v)' box whose output is labelled '-1 ... 1'.
The siamese setup that trains sentence embeddings so cosine similarity tracks meaning. It needs pairs carrying a similarity label, which is a different dataset from the single-text-plus-class data a classifier wants. Image: UKPLab/sentence-transformers · Apache-2.0 repository
№ 04

PhilPapers Papers Summarized Labeled

Text
Size
3,776 documents in JSON format, including titles, summaries, multi-label labels
Licence
MIT

3,776 PhilPapers philosophy documents in JSON, each with a title, an LLM-generated two to three sentence summary, and multi-label tags across 17 philosophical schools. A rare labelled humanities corpus.

What to train on it

Training multi-label classification on academic texts, developing automatic summarisation in philosophy, and exploring trends across schools of thought.

Source: innovatiana.com

№ 05

Civil Comments - Corpus of moderated comments annotated for toxicity

Text
Size
Approximately 2 million text comments in JSON format, with toxicity labels and types of abuse
Licence
CC0-1.0

Roughly two million English public comments in JSON, gathered from a news-site commenting plugin between 2015 and 2017, labelled for overall toxicity and for specific kinds of abuse. Public domain and large.

What to train on it

Training toxicity detection and automated moderation, analysing hostile online interaction dynamics, and testing multi-class and multi-label classification on long texts.

Source: innovatiana.com

№ 06

Chinese Sentiment Analyze

Text
Size
182,762 examples of Chinese text (reviews and social posts), JSON/CSV format
Licence
MIT

182,762 Chinese-language examples in JSON and CSV, combining e-commerce product reviews with Weibo social posts, each labelled positive, neutral or negative. Covers both commercial and social registers.

What to train on it

Training sentiment classification in Mandarin, developing opinion analysis for commercial or social applications, and testing multilingual model robustness on everyday Chinese.

Source: innovatiana.com

№ 07

Women's E-Commerce Clothing Reviews

Text
Size
23,486 rows in CSV format, text and categorical data
Licence
CC0: Public Domain

23,486 CSV rows of anonymised online clothing reviews combining free text with ratings, customer ages, recommendation flags and product categories. Compact, clean and a long-standing teaching set for applied NLP.

What to train on it

Training sentiment analysis or review classification, studying customer experience by age or product category, and exploring BERT, TF-IDF or Word2Vec on real data.

Source: innovatiana.com

Four charts in a two-by-two grid. Top left, 'length distribution': a histogram of comment lengths up to 2,000 characters, mostly green (not toxic) with a thin red (toxic) band. Top right, 'toxicity ratio' by length: a line starting near 0.17 for the shortest comments and settling around 0.05. Bottom left, the same histogram for comments containing 'gay', where red dominates the short comments. Bottom right, its toxicity ratio: near 0.88 for the shortest comments, falling, then swinging widely as the counts thin out.
Class balance in the Wikipedia talk-page toxicity data, from Conversation AI's dataset-bias notebook: about one comment in ten is labelled toxic overall (top), but 55% of comments containing 'gay' are, and most of the short ones (bottom). A classifier trained on that learns the identity term as the signal, the failure the identity labels in Civil Comments were later collected to measure. Image: conversationai/unintended-ml-bias-analysis · Apache-2.0 repository
№ 08

Twitter Sentiment Analysis Dataset

Text
Size
Approximately 1.6 million annotated tweets, in CSV format
Licence
Use subject to the Twitter API terms of use. Verification required for commercial uses

About 1.6 million English tweets in CSV labelled positive or negative automatically, from emoticons rather than by annotators (only the 498-tweet test set adds a neutral class). Note the licence defers to Twitter's (now X's) API terms rather than granting an open licence, so commercial use needs verification.

What to train on it

Training sentiment classification on short texts, analysing trends and opinion on social networks, brand monitoring, and improving moderation or opinion summarisation.

Source: innovatiana.com

№ 09

Sentiment Analysis for Mental Health

Text
Size
Approximately 51,000 texts (statements) annotated into 7 categories, CSV/JSON format
Licence
Open Database License (ODbL) or equivalent free license (please check before using)

Over 51,000 statements gathered from platforms such as Reddit and Twitter in CSV and JSON, each labelled with one of seven mental health states. Its catalogue page itself advises checking the licence before use.

What to train on it

Training classification of mental states from text, developing psychological support chatbots, and emotional analysis to detect mental health trends.

Source: innovatiana.com

№ 10

Amazon Polarity

Text
Size
Around 4 million reviews (train + test), JSON format with title, content and binary label fields
Licence
Apache 2.0

Around four million Amazon customer reviews in JSON with title, content and binary positive-or-negative label, spanning eighteen years of commerce. Large and permissively licensed, it is the workhorse sentiment classification corpus.

What to train on it

Training sentiment classification for e-commerce, improving customer feedback analysis and recommendation, and testing NLP robustness on large real-world volumes.

Source: innovatiana.com

Banking77banking_data/train.csv · lines 2, 1483, 8961
textcategory
I am still waiting on my card?card_arrival
I'm panicking! I lost my card! Help!lost_or_stolen_card
If I exchange currencies on a frequent basis can I receive a discount?exchange_charge
Banking77 as it ships: one customer query, one of 77 intent labels. Classes this fine-grained — a card that has not arrived against one that was lost — are what make it the hard intent-classification benchmark. From the dataset: PolyAI-LDN/task-specific-datasets · banking_data/train.csv · CC BY 4.0

How this list was put together

Selected by searching Dataset Finder for datasets whose own documentation describes them as suitable for text classification; 93 matched, and the shortlist favours those with an unambiguous licence and the largest published scale.

Sizes and licences come from each dataset's documentation, vague wording included; where we found a re-upload's licence contradicting the original release, the entry gives the original terms. A dataset only appears here if its own description supports the task — we would rather publish a short list than pad one.

Stop hunting. Start searching.

The datasets listed on this page are in Dataset Finder, alongside tens of thousands more — searchable in plain language.

Search Dataset Finder →