AI 101 — What is data labeling?

Share
AI 101 — What is data labeling?

Data labeling is the work of attaching an answer to a raw example — this photo shows a cat, this reply is rude, this answer is better than that one — so a model has something to be graded against while it learns. Remove the labels and a model is just a very large pile of text with no idea which parts are right.

Almost everything people call "AI training" is really two jobs: showing the model examples, and telling it what the correct response to each one is. The first job is mostly scraping. The second is labeling, done by people — millions of them over the past fifteen years, clicking boxes, ranking pairs of sentences, and now writing model answers themselves.

The classic picture is contractors drawing rectangles around pedestrians so a car can learn to see them. That work still exists, and it is now the cheapest, most automated corner of a business that has moved sharply upmarket.

Why it matters right now

The labeling industry's biggest company is now half-owned by one of its customers. Meta paid $14.3 billion for a 49% stake in Scale AI in June 2025, valuing the data-labeling startup at $29 billion and bringing its chief executive into Meta's AI team, Reuters reported. Rivals reacted immediately: Google paused work with Scale within hours and OpenAI said it was already winding its own work down, per analysis published by Investment Brief.

The spending did not disappear; it changed shape. The same analysis of Anthropic's securities filing puts its payments to data vendors at roughly $1 billion in the nine months to September 2026 — about three times its spend for all of 2025, and around 5% of what it spends on compute.

What that billion buys is not boxes around cats. It buys doctors, lawyers and accountants performing a slice of their own job so a model can be trained on it, and grading the model's attempts afterward. Labs increasingly rent the whole exercise as a simulated workplace — a "reinforcement learning environment," in the industry's term — where an agent tries a task and gets a score. A standard environment runs roughly $200 to $2,000 per task, and a faithful replica of a company's workflow costs about $300,000 to build, according to a vendor survey published by CleverX.

The mental model: an example, and a verdict

Every labeled example is a pair: the thing itself, and the judgment about it. Learning is the model adjusting until its own guesses match the judgments often enough to be useful.

That judgment comes in three grades, and the price rises with each.

The first is a category: cat, not dog; spam, not invoice. This tier is cheap, well understood, and increasingly done by a model first and checked by a person second.

The second is a preference: here are two answers, pick the better one. A human ranking millions of pairs is how a raw text predictor becomes a helpful assistant, the process we described in AI 101 — What is RLHF?.

The third is a task with a rubric: an expert does the work, then writes down what a correct answer looks like, line by line. This is where the money is now, because a rubric written by a tax partner is worth more than ten thousand clicks by someone reading a one-page instruction sheet.

One useful wrinkle: a label does not have to come from a person clicking. A unit test that passes or fails is a label. A scoring script is a label. It is why coding agents improve so quickly — a repository arrives with its own answer key built in.

The analogy: cooking school

A student cook can read recipes all day and still plate something inedible. Learning to cook needs a taster: too salty, too long in the oven, that one is right. The taster is the label.

Then the student needs a real service — orders coming in, dishes going out, a head chef scoring each plate at the pass. That is the environment: many small decisions in sequence, each one affecting the next, with a score at the end. A written exam cannot teach service, which is exactly why labs stopped buying static datasets for agent training and started renting kitchens instead.

And the analogy carries the central problem. If two tasters disagree about the same soup, the student learns noise. Labeling teams measure this as agreement between labelers, and a dataset where experts agree only 70% of the time caps how good the model trained on it can get. The guidelines — the document defining what counts as correct — are often the most valuable artifact in the pipeline.

Common misconceptions

"The model learns on its own from the internet." The internet supplies examples, not answers. Someone or something has to say which response is right, and every frontier model's personality is the sum of those judgments.

"Labeling is unskilled clicking." The bottom tier is. The top tier is people with professional credentials being paid for exactly the expertise they trained for, plus the people who write and test the rubrics.

"Synthetic data made labeling obsolete." Synthetic data is generated from patterns a model already learned — it recycles judgment rather than adding new judgment. Train mostly on it and quality degrades, the failure mode we covered in AI 101 — What is model collapse?. Human labels stay the reference point that synthetic data is checked against.

"More labels always means a better model." A large, noisy answer key teaches the wrong lesson efficiently. Teams keep a small set of examples labeled to a high standard to test against — the same idea as the held-out sets in AI 101 — What is fine-tuning?.

"It is all overseas piecework." That was true of the first era, when image tagging moved to low-cost crowdsourcing platforms. The growth now is expert work: engineers, clinicians, traders and lawyers paid by the hour to do their own jobs on camera, for a model that will eventually imitate them.

What to watch

Whether environments replace datasets entirely, and who owns the rubrics. A lab that trains on your firm's workflow, scored by your firm's standard, is buying something closer to institutional knowledge than to a training file — and that is harder to walk away from than a vendor contract.

If the valuable labels now come from credentialed experts, what happens to the millions of people who built the last decade of AI data? Write to us via our contact page.

Related reading: AI 101 — What is RLHF? · AI 101 — What is model collapse? · AI 101 — What is fine-tuning?

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

AI 101 — What is a knowledge graph?

AI 101 — What is a knowledge graph?

A knowledge graph is a database that stores facts as things and the relationships between those things — a company, a person, a part, a drug — rather than as rows in a table or chunks of text, so software can follow the connections instead of just matching words. Google put the idea on the map in May 2012, describing a graph that understood real-world entities and how they relate — the company's framing was "things, not strings" — and said the launch version held more than 500 million objects…

AI 101 — What is generative engine optimization (GEO)?

AI 101 — What is generative engine optimization (GEO)?

Generative engine optimization is the practice of shaping a web page so that AI answer engines — ChatGPT, Google's AI Overviews, Gemini, Claude — quote it and link to it when someone asks a question. It is search engine optimization for a search box that writes an answer instead of handing you ten links. You have probably met it without the name. Someone asks ChatGPT a question about your industry and the reply cites four websites, three of them competitors.

Tempo ties AI spend to Jira work items with Workforce Intelligence

Tempo ties AI spend to Jira work items with Workforce Intelligence

Enterprises can finally answer a question they've been too embarrassed to ask aloud: exactly what did that AI token bill buy? Tempo, the Atlassian ecosystem's planning-and-portfolio software maker, launched Workforce Intelligence on Tuesday, an app that connects AI usage and its cost to the specific Jira ticket the work was done on. Tempo launched Workforce Intelligence, an Atlassian Marketplace app that attaches AI activity and its cost to the individual Jira work item it supported.

How to — tell if an AI model has quietly changed

How to — tell if an AI model has quietly changed

The model you call today is not necessarily the model you tested. Providers update the served system in place — a fine-tune, a new safety classifier, a rewritten system prompt, a different route — and your request still returns 200 with a perfectly normal-looking answer. You can catch that on your own inputs, in about an afternoon of setup and a few minutes a week, so a provider update shows up as a dated event in your logs instead of a support ticket three weeks later. The change is not…

Claude Message Threads: 35% faster first token, same token bill

Claude Message Threads: 35% faster first token, same token bill

Anthropic's Messages API has been stateless since it launched: every turn resends the entire transcript, tool results and screenshots included. That is fine until an agent loop passes half a million tokens, at which point the resend becomes the slowest part of the request. As of this week the API can hold the conversation on its own side instead. Anthropic put message threads into public beta on Monday, and the number attached to it is a median 35% cut in time to first token for requests…