Long context costs 3.6x to 7x more per token. Compaction holds up

Two numbers a buyer never sees on a pricing page: what a long prompt actually costs the seller, and how little accuracy the buyer gives up by not sending one.

Serving an open-weight model at a 256K-token context instead of 16K costs 3.6 times more per generated token for GLM 5.2, 5.6 times more for Kimi K2.6 and 7 times more for Qwen 3 Coder Next, according to a 31-page NVIDIA technical report posted to arXiv on November 3. The paper, "You don't need long context," was written by Peter Belcak and eleven co-authors at NVIDIA Research. They measured sustained decode throughput at each context window size on H100 nodes, converted it to dollars at an on-demand price of $3.85 per GPU-hour, and plotted the cost of generating one million output tokens against context length. The line goes up steadily from 16K to 256K: Kimi's curve climbs to about $10 per million tokens, Qwen 3 Coder Next's to roughly $1.50.

The mechanism is not a vendor's greed, it is the attention math underneath every deployed model. Generating a token requires reading the whole live context, so cost grows with depth: the thousandth token is cheap, the two-hundred-thousandth is not. The report notes that this is the one cost driver retail pricing deliberately flattens — APIs quote a single price per million tokens whether you send 900 tokens or a 300-page contract — which means short requests are quietly subsidizing long ones. NVIDIA's framing is blunt: the 3.6x to 7x spread "rivals or exceeds" the gains from aggressive quantization or structured sparsification, the optimizations teams spend quarters chasing.

The second half of the paper is the part practitioners will argue about. Across six long-horizon benchmarks — SWE-bench Verified, Terminal-Bench 2.1, SWE-bench Multilingual, FRAMES, BFCLv3 and Tau2Bench — and seven long-input ones including LongBench v2, LOONG, AA-LCR, needle-in-a-haystack and RULER, the authors ran frontier models with two compaction regimes instead of full history. End-of-conversation compaction summarizes state at a threshold and starts a fresh conversation; the results landed between 3.7 points worse and 5.6 points better for Claude Opus 4.5, and between 3.3 points worse and 4.4 points better for GPT 5.6 Sol, depending on benchmark and how hard the context was squeezed. Estimated cost savings ran from 2.2x to 8.8x for Opus and 2.4x to 5.0x for Sol.

The long-input results are stranger and more interesting. With selective reading — the harness greps an input file instead of pasting it — savings ranged from 6.5x to 54.6x for Opus 4.5 and from 4.4x to 233.3x for GPT 5.6 Sol, and accuracy did not fall. On the needle-style tasks it rose, by as much as 40 points, which is less a triumph for compaction than an indictment of the haystack: a target sentence is easier to find in 16K tokens than buried in 500K of filler.

Then the regime the authors clearly prefer. Single-action compaction allows the model at most one action per context, so every step starts nearly clean; state lives in files and in a written handoff rather than in the window. Under that regime GPT 5.6 Sol reached 82.4% on SWE-bench Verified and 84.3% on Terminal-Bench 2.1, solving 75 of 89 tasks, with median peak live context of just 7,000 to 9,000 tokens and a maximum that fell from 130,000–200,000 tokens to 15,000–37,000. From that evidence the authors sketch "semantic state space models" — models post-trained to maintain an explainable natural-language state and hand off to a fresh inference instance, the way a colleague would brief a relief shift.

The take worth having is not "compaction works" — anyone running an agent harness learned that in practice this year, and we have covered the mechanics in our explainer on what context engineering actually is. It is that flat per-token pricing has been hiding a cost curve that is steeper than the price cuts vendors keep announcing. If a 256K prompt costs the provider up to seven times what a 16K prompt costs and both are billed at the same rate, then long-context usage is being paid for out of margin, and the pricing page is a promotional instrument rather than a cost model. That is the kind of gap that closes in a quarter: expect context-length tiers, or per-request surcharges above some threshold, at whichever provider blinks first.

The counter-case deserves room, because the paper's own numbers provide it. Compaction is lossy summarization, and the tasks where it hurts are exactly the ones with no lexical handle — a clause buried in a 400-page contract, a contradiction between page 12 and page 340 — where the summarizer has no way to know which sentence will matter. The LongBench v2 delta for Opus 4.5 was 0.4 points negative, which is flat, not a win. Single-action compaction also multiplies turns: one action per context means many more calls, and the token math does not price the orchestration, the latency, or the engineer who has to debug a relay race of handoffs. And NVIDIA sells the GPUs; a result showing that customers can serve more tokens per GPU-hour is good engineering and also convenient for the company publishing it.

The practical read for anyone shipping an agent this quarter: cap the window, compact at a threshold you chose on purpose, and measure the accuracy delta on your own eval rather than trusting a published range, because the spread here is eight points wide in both directions. Keep full context for the single-pass case where the model must read everything at once, and stop treating a million-token window as a feature you are obligated to fill.

Dropbox is buying Tigris Data for $350 million to build a storage layer for AI agents, its largest acquisition to date, TechCrunch reported. The deal, announced November 5 and disclosed in an 8-K as agreed on November 3, is expected to close in the fourth quarter and sits under a new product called DBX Agent Cloud, which starts with standalone S3-compatible object storage. Tigris was founded by former Uber engineers who built the ride-hailing company's storage infrastructure, and raised a $25 million Series A last year; Dropbox says it has no plans to cut headcount as part of the acquisition, after cutting 9% of its workforce last month.

The strategic logic is narrower than "Dropbox does AI," and more interesting. Agents need somewhere to put state that survives a session — training sets, model artifacts, multimodal files, and the scratch space an agent writes to between tasks — and that space has to work across clouds and compute providers. Dropbox is not competing on models; it is selling the content layer beneath them, which is the only part of the stack where a two-decade-old file company still holds an advantage. Its own AI rollout has reached more than 70,000 teams on Standard and Advanced plans in the US, UK and Australia, with model switching inside a session and an action log for every change. The filesystem and content-processing pieces of DBX Agent Cloud are slated to follow the object storage launch.

What to watch: whether a major API publishes context-tiered pricing before the next model cycle, and whether Dropbox ships the filesystem half of Agent Cloud on schedule.

If you run agents in production, what context window do you actually give them — and have you measured what the extra tokens buy? Write to us via our contact page.