AI 101 — Active vs total parameters in MoE models

Every big model now ships with two parameter counts: total, which is how many weights it owns, and active, which is how many it uses to produce a single token. The gap between them — often thirtyfold — is the whole reason a trillion-parameter model can be served at all.
The architecture that creates the gap is called mixture of experts (MoE). Instead of one large network doing all the work, the model holds dozens or hundreds of smaller sub-networks — the "experts" — and a router that sends each token (a chunk of a word) to only one or two of them. The model is enormous on paper and does the arithmetic of something much smaller.
That is why Meituan's LongCat-2.5 can claim 1.6 trillion parameters while activating about 48 billion of them per token, according to the model card. Two numbers, one model, roughly 33 times apart. Here is what the difference means, and which of the two you should be reading.
Why it matters right now
Almost every large model released in the last two years is a mixture of experts, and the ones that are not tend to say so loudly. DeepSeek-V3 carries 671 billion parameters and activates 37 billion per token, the technical report states. Mistral's Mixtral 8x7B has 47 billion total and uses 13 billion per token, per its paper. Meituan's release is the same architecture at a bigger scale, and its serving recipe — 194 Huawei accelerators for a trillion-parameter-class model — only makes sense once you know that most of those parameters sit idle on any given word.
Model cards lead with the big number because it is the one that sells. The small number is the one that decides your bill.
The mental model
A transformer is a stack of identical layers, and each layer does two jobs: attention (work out which earlier words matter) and a feed-forward block (think about what was found). The feed-forward block is where most of the parameters live — often two-thirds of the model.
A mixture of experts replaces that block. Instead of one feed-forward network per layer, you keep N of them side by side, and you put a router in front: a small linear layer that scores every expert for the token at hand, keeps the top one or two, and adds their outputs together in proportion to the scores. Attention, embeddings and the output layer stay shared.
So the parameter count splits cleanly. Total = shared parts + N experts. Compute per token = shared parts + the one or two experts the router picked. That is how a model can have 256 experts per layer and still run at a speed set by a fraction of them — and why "mixture of experts" is sometimes called sparse: for each token, most of the network is skipped.
The hospital analogy
Picture a large hospital with 64 specialists on staff and a triage desk at the door. Every patient is seen by the triage nurse, who reads the chart and sends them to two specialists — say cardiology and endocrinology — who consult together and hand back one combined opinion.
Two different costs fall out of that arrangement, and conflating them is where most people go wrong. The hospital pays all 64 salaries whether or not anyone walks in, and it must keep every department stocked and staffed: that is memory. Each patient, meanwhile, consumes two consultations, not 64: that is compute. The hospital is expensive to run and cheap per visit, all at once.
Common misconceptions
"1.6 trillion parameters means it thinks with 1.6 trillion." No. Per-token reasoning depth tracks the active count much more closely. What the extra parameters buy is breadth — more stored patterns to route to — and the ability to reach a model of that size's quality at a fraction of the compute. Nobody gets a dense 1.6-trillion-parameter model's behaviour at 48 billion; the honest claim is that you get closer to the big model than a 48-billion-parameter model would, for closer to the small model's cost.
"MoE saves memory." It does the opposite. Every expert has to be loaded and ready, because the router can call any of them on the next token. Hugging Face's explainer puts it plainly: mixture-of-experts models require high VRAM because all experts sit in memory, and running Mixtral means having room for a 47-billion-parameter dense model even though only 13 billion are used per token. If you are comparing against a dense model of the same total size, you save nothing on hardware; you save on speed.
"Each expert is a domain specialist." Tempting, and mostly wrong. Google's ST-MoE researchers looked at which tokens went where and found shallow specialisation — a punctuation expert, a proper-noun expert — rather than a "biology expert" and a "poetry expert". The router is clustering surface patterns, not hiring professors.
"This is new." The idea dates to a 2017 Google paper on sparsely-gated layers, and the Switch Transformer in 2021 pushed it to trillion-parameter models with one expert per token. What changed is not the concept but the economics: training a sparse model is now cheaper than training a dense one of the same quality, and inference providers can quote per-token prices that look like a mid-size model's.
"Sparsity is free." The catch sits in the plumbing. Experts live on different chips, so tokens have to be shipped across the network mid-calculation, and the router has to be kept from dumping everything on one popular expert. That load-balancing problem is why sparse training is as much a networking discipline as a machine-learning one.
Where it lands
The practical reading is a shopping rule. Compare models on active parameters when you care about tokens per second and price, and on total parameters when you care about what hardware can hold the weights. LongCat-2.5 is the clean example: a 3.56 TB checkpoint in BF16, cut to about 1.34 TB at 4 bits — the same compression trick we walked through in what model quantization does, and the reason a model this size can be served at all. If you want the background on what a parameter actually is, start with what model parameters are.
The bigger point is that the parameter count stopped being a size measurement and became a billing construct. Two models can both say "1.6 trillion" and behave like very different machines.
Related reading
- AI 101 — What is a small language model? — the other end of the scale, and why narrow models win on repeat work.
- AI 101 — What is an LLM router? — the same routing idea, one level up: choosing between whole models instead of experts.
Would you rather have a model with more parameters or a cheaper token? Write to us via our contact page.







