The Take — Proof-of-inference is a receipt, not a control

The most important line in Anthropic's provable-inference release is not the 110 tokens per second. It is a sentence further down the same page, in the section on pacing: proving that a provider's hardware is serving inference, Anthropic writes, "does not establish that the hardware is not, e.g., training on the side."

That is the question. Every compute-pacing proposal in Washington and Brussels is built to answer it — is this cluster doing the inference you said, or is it training something else? The best cryptographic primitive anyone has published this year says, in its own documentation, that it does not answer it. I think FlashProve is a real advance, and I think it is about to be sold as a compliance regime when what it actually is is an audit instrument. The difference is not academic. It is the difference between a control a regulator can write into a rule and a receipt a company hands over when it chooses to.

What actually shipped

Anthropic's researchers, with academic collaborators from EPFL, MIT and Succinct, published a system called FlashProve on November 3: a zero-knowledge proof that a specific set of model weights produced a specific set of output tokens, without revealing the weights. For Llama 4 Maverick, a 400-billion-parameter mixture-of-experts model at 4-bit, a single H100 generates proofs at about 110 tokens per second, each proof is roughly 19 megabytes, and a verifier on an ordinary CPU checks one in 17 seconds on 24 threads.

The progress is not marginal. The previous published proof of that same model, VerInf, took 14.3 hours for 1,000 tokens and produced a 93.6-gigabyte proof that took most of a day to verify. FlashProve does it at 110 tokens per second with a proof you can check on a laptop-class CPU. That is three orders of magnitude in roughly a year, and it should update anyone's priors about what cryptography can do to model accountability.

Now the parts that decide whether it can carry a policy.

The proof is about an integer model under greedy decoding, over a window of 1,024 tokens. Anthropic is explicit that a floating-point implementation or sampled decoding "could depart at some position and no longer exactly match the computation proved." No production provider serves greedy decoding over 1,024-token windows; sampling is the product, and agent sessions run to hundreds of thousands of tokens. The system also assumes a mixture-of-experts architecture, and it leaves side-channel leakage — timing, for instance — out of scope. The fast C++ verifier that a customer would actually use carries no formal guarantee; the Lean-verified verifier is the slow reference.

Then the cost. Proving a batch of 24 requests took 223 seconds of H100 time, which is about 72 times the GPU time vLLM needed for the same batch, and roughly 94 times at the highest load Anthropic tested. As our roundup of the release noted, that is far too expensive to run on all traffic. So the shape of the thing is fixed: you prove a sample, after the fact, and someone checks it.

An audit is a sampling regime with consequences

That is not a criticism of the cryptography. It is the observation that turns a research prototype into a policy question, and almost nobody is asking it.

Audits work because three things exist together: a sampler who is not the audited party, a standard for what counts as a pass, and a consequence for a failure. FlashProve supplies the second one beautifully — the proof either verifies or it doesn't, and the Lean formalisation means the security claim itself is machine-checkable, with an interactive visualizer so a human can read the statement being proved rather than trusting the proof. That is a serious contribution to governance, arguably the most useful part of the release.

It supplies neither of the other two. Nothing in the design lets a third party demand proofs for a random slice of a provider's live traffic; the prover produces proofs, and the prover is the party with something to hide. A proof you get because you asked politely and were not refused is a disclosure, not an audit. And no one has written down what a failed proof triggers — a fine, a suspension, a disclosure, a shrug. The efficiency that makes the system work makes this worse: producing a proof costs 94 times the inference, checking one costs seconds, so the expensive side is the side that decides whether a proof exists at all.

The strongest case for the other side

The steelman is good, and parts of it I accept outright. This is a research prototype, published with code, a technical report and Lean proofs under a permissive licence, explicitly to be improved — the opposite of a vendor benchmark. Three orders of magnitude in a year is not a plateau; the next factor of ten is plausible, and if context length and sampled decoding follow, most of my objection evaporates.

Second, Anthropic already met its own roadmap commitment by another route. Its Frontier Safety Roadmap promised a prototype by September 30 for signing inference traffic in a way that makes outputs attributable to specific weights, and the company says it hit that goal with a signing-and-recompute design on a trusted server. FlashProve is a parallel bet, not a substitute — which is more honest than most safety announcements manage.

Third, the narrow claim it does make is a live problem today. A routing service that silently swaps in a cheaper model, or a compromised server injecting tokens, is a real fraud with real victims, and "which weights answered you" is exactly the fact nobody can currently check. Getting that to 110 tokens per second is worth something on its own.

And regulators do run sampled regimes: tax authorities audit a fraction of returns, food inspectors visit a fraction of kitchens, and both work. Requiring a proof on every token was never the plan.

Why I still think the framing is wrong

Because policy is written at the altitude of the headline, and the headline is "verifiable inference." The qualifiers — integer arithmetic, greedy decoding, 1,024 tokens, MoE only, no side-channel guarantee, no answer to the training-on-the-side question — live in a limitations list that no statute quotes. The risk frontier is also moving in the one direction the proof cannot follow: long-horizon agents, sampled outputs, contexts in the hundreds of thousands of tokens. The primitive is strongest exactly where the stakes are lowest and weakest where they are rising.

Anthropic's own IPO filing is the useful test case. The company told investors that recursive self-improvement may be near and that it might not detect it in time. Proof-of-inference is the technology that would let an outsider check whether the model being served is the model that was approved — and it is one of the few concrete things in that filing a regulator could actually ask for. If the answer to "can you prove it?" turns out to be "yes, for a 1,024-token greedy sample of a model we chose to prove," then the filing's risk factor stays a risk factor.

My prediction, for the record: within a year, some lab cites provable inference as evidence of compliance with a voluntary commitment, and the thing being evidenced will be that the weights matched — not that the commitment was honoured. That is how a good cryptographic tool becomes a compliance ornament, and it will not be the cryptographers' fault.

What would change my mind

Four things, in order of weight. A proof that covers sampled decoding at production context lengths — 100,000 tokens and up — at under ten times the cost of plain inference, which would make continuous proving plausible instead of sampled. A published audit endpoint: an accredited third party can request proofs for a random slice of live traffic, and failures are published. A regulator that writes down the sampling rate and the consequence for a failed proof, and a provider that volunteers for it before being asked. And a refusal on the record — a lab declining an auditor's proof request would prove, faster than any paper, that the tool matters.

Until then, the honest description is this: Anthropic has built the cheapest receipt in the history of computing, and a receipt proves what you bought, not that the shop is honest.

If a proof can only be produced by the party with something to hide, is it evidence or marketing? Write to us via our contact page.