AI 101 — Active vs total parameters in MoE models

Share
AI 101 — Active vs total parameters in MoE models

Every big model now ships with two parameter counts: total, which is how many weights it owns, and active, which is how many it uses to produce a single token. The gap between them — often thirtyfold — is the whole reason a trillion-parameter model can be served at all.

The architecture that creates the gap is called mixture of experts (MoE). Instead of one large network doing all the work, the model holds dozens or hundreds of smaller sub-networks — the "experts" — and a router that sends each token (a chunk of a word) to only one or two of them. The model is enormous on paper and does the arithmetic of something much smaller.

That is why Meituan's LongCat-2.5 can claim 1.6 trillion parameters while activating about 48 billion of them per token, according to the model card. Two numbers, one model, roughly 33 times apart. Here is what the difference means, and which of the two you should be reading.

Why it matters right now

Almost every large model released in the last two years is a mixture of experts, and the ones that are not tend to say so loudly. DeepSeek-V3 carries 671 billion parameters and activates 37 billion per token, the technical report states. Mistral's Mixtral 8x7B has 47 billion total and uses 13 billion per token, per its paper. Meituan's release is the same architecture at a bigger scale, and its serving recipe — 194 Huawei accelerators for a trillion-parameter-class model — only makes sense once you know that most of those parameters sit idle on any given word.

Model cards lead with the big number because it is the one that sells. The small number is the one that decides your bill.

The mental model

A transformer is a stack of identical layers, and each layer does two jobs: attention (work out which earlier words matter) and a feed-forward block (think about what was found). The feed-forward block is where most of the parameters live — often two-thirds of the model.

A mixture of experts replaces that block. Instead of one feed-forward network per layer, you keep N of them side by side, and you put a router in front: a small linear layer that scores every expert for the token at hand, keeps the top one or two, and adds their outputs together in proportion to the scores. Attention, embeddings and the output layer stay shared.

So the parameter count splits cleanly. Total = shared parts + N experts. Compute per token = shared parts + the one or two experts the router picked. That is how a model can have 256 experts per layer and still run at a speed set by a fraction of them — and why "mixture of experts" is sometimes called sparse: for each token, most of the network is skipped.

The hospital analogy

Picture a large hospital with 64 specialists on staff and a triage desk at the door. Every patient is seen by the triage nurse, who reads the chart and sends them to two specialists — say cardiology and endocrinology — who consult together and hand back one combined opinion.

Two different costs fall out of that arrangement, and conflating them is where most people go wrong. The hospital pays all 64 salaries whether or not anyone walks in, and it must keep every department stocked and staffed: that is memory. Each patient, meanwhile, consumes two consultations, not 64: that is compute. The hospital is expensive to run and cheap per visit, all at once.

Common misconceptions

"1.6 trillion parameters means it thinks with 1.6 trillion." No. Per-token reasoning depth tracks the active count much more closely. What the extra parameters buy is breadth — more stored patterns to route to — and the ability to reach a model of that size's quality at a fraction of the compute. Nobody gets a dense 1.6-trillion-parameter model's behaviour at 48 billion; the honest claim is that you get closer to the big model than a 48-billion-parameter model would, for closer to the small model's cost.

"MoE saves memory." It does the opposite. Every expert has to be loaded and ready, because the router can call any of them on the next token. Hugging Face's explainer puts it plainly: mixture-of-experts models require high VRAM because all experts sit in memory, and running Mixtral means having room for a 47-billion-parameter dense model even though only 13 billion are used per token. If you are comparing against a dense model of the same total size, you save nothing on hardware; you save on speed.

"Each expert is a domain specialist." Tempting, and mostly wrong. Google's ST-MoE researchers looked at which tokens went where and found shallow specialisation — a punctuation expert, a proper-noun expert — rather than a "biology expert" and a "poetry expert". The router is clustering surface patterns, not hiring professors.

"This is new." The idea dates to a 2017 Google paper on sparsely-gated layers, and the Switch Transformer in 2021 pushed it to trillion-parameter models with one expert per token. What changed is not the concept but the economics: training a sparse model is now cheaper than training a dense one of the same quality, and inference providers can quote per-token prices that look like a mid-size model's.

"Sparsity is free." The catch sits in the plumbing. Experts live on different chips, so tokens have to be shipped across the network mid-calculation, and the router has to be kept from dumping everything on one popular expert. That load-balancing problem is why sparse training is as much a networking discipline as a machine-learning one.

Where it lands

The practical reading is a shopping rule. Compare models on active parameters when you care about tokens per second and price, and on total parameters when you care about what hardware can hold the weights. LongCat-2.5 is the clean example: a 3.56 TB checkpoint in BF16, cut to about 1.34 TB at 4 bits — the same compression trick we walked through in what model quantization does, and the reason a model this size can be served at all. If you want the background on what a parameter actually is, start with what model parameters are.

The bigger point is that the parameter count stopped being a size measurement and became a billing construct. Two models can both say "1.6 trillion" and behave like very different machines.

Related reading

Would you rather have a model with more parameters or a cheaper token? Write to us via our contact page.

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

AI 101 — What is gradient descent?

AI 101 — What is gradient descent?

Gradient descent is the loop almost every AI model runs to get better: measure how wrong its answers are, work out which direction would make the error smaller, and nudge every one of its numbers a little that way. Then do it again — millions of times. That is the whole idea. It is older than the computer, it fits on a napkin, and it is still what happens inside a training run that costs tens of millions of dollars and burns through a power station's worth of electricity. Why it matters…

AI 101 — What is a reasoning model?

AI 101 — What is a reasoning model?

A reasoning model is a language model trained — usually with reinforcement learning — to write a long chain of thought before it answers, so it can spend more computation on a hard problem instead of firing back the most likely next sentence. The label arrived with OpenAI's o1 in September 2024. Before it, a chatbot answered in one pass: read the question, predict the reply token by token. o1 added a visible delay and a trail of working behind the answer.

Long context costs 3.6x to 7x more per token. Compaction holds up

Long context costs 3.6x to 7x more per token. Compaction holds up

Two numbers a buyer never sees on a pricing page: what a long prompt actually costs the seller, and how little accuracy the buyer gives up by not sending one. Serving an open-weight model at a 256K-token context instead of 16K costs 3.6 times more per generated token for GLM 5.2, 5.6 times more for Kimi K2.6 and 7 times more for Qwen 3 Coder Next, according to a 31-page NVIDIA technical report posted to arXiv on November 3.

Today in AI — November 7, 2026

Today in AI — November 7, 2026

The money kept moving the day after a market rout — a $1.2 billion drone stake, a $200 million video-model round, Meta's strategy chief leaving for an inference startup — while a chip-export sieve in China and an exploit hiding in a plain text file supplied the day's harder reading. Models & Research * The man who proposed the Exponential Time Hypothesis calls OpenAI's disproof "very shocking news." Russell Impagliazzo, who formulated the hardness assumption with Ramamohan Paturi in 1999…

AI 101 — What is GRPO (group relative policy optimization)?

AI 101 — What is GRPO (group relative policy optimization)?

GRPO is the reinforcement-learning method behind most open reasoning models: instead of training a second neural network to judge how good each answer is, it has the model answer the same question many times and rewards whichever answers beat the group's own average. The name is an initialism nobody says out loud — group relative policy optimization — but the idea fits in a sentence, and once you have it a lot of recent AI news stops sounding like alchemy.

AI 101 — What is model collapse?

AI 101 — What is model collapse?

Model collapse is what happens to a generative AI model that learns mostly from other AI models: it slowly stops representing the full range of real data, and the loss is not recoverable by training longer. That is the plain version. The precise version, from the 2024 study that named the effect, is that "indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear" — published in…