Deep Dive — Clover 35B: a diffusion LLM at Qwen parity, 680 tok/s

DiffuSpace released Clover-35B-A3B-Preview on Tuesday: a 35-billion-parameter mixture-of-experts diffusion language model under Apache 2.0, with weights on Hugging Face and ModelScope, a 128K context window, a thinking mode, and serving paths in SGLang and llama.cpp. It reports 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 65.3 on τ³ Overall, plus 680 tokens per second on a single GB300 at BF16, batch size one, according to the launch blog.

That is the first open-weight diffusion model put directly against the autoregressive model it was trained from and asked to win. The winning margin on the headline coding benchmark is 0.2 of a point, and the baseline is a re-run. Both facts are more interesting than the launch copy, and neither makes the release unimportant.

What DiffuSpace actually shipped

The team is the commercial continuation of the University of Hong Kong's NLP group, which shipped Dream 7B in April 2025 — the model that first showed a diffusion language model could hold its own against autoregressive models at small scale. The Dream family has passed 2.7 million downloads, and DiffuSpace is what several of those authors built next. Clover is their first release, and it is roughly five times the size of Dream.

The spec sheet is a Qwen derivative, openly stated: Clover is built on Qwen3.6-35B-A3B, keeping that model's 40 layers, 256 routed experts with top-8 routing plus one shared expert per layer, and its hybrid Gated DeltaNet / full-attention backbone. What DiffuSpace added is block diffusion on top: instead of writing one token and feeding it back, Clover fills a block of 32 masked positions, resolves the ones it is confident about in parallel, and only then moves on. The Hugging Face defaults set the confidence threshold at 0.95, so a block closes when the model is sure, not when a step count runs out.

The weights are about 69 GB in BF16, so this is a single-90GB-card model or a quantised one, not a laptop download. Apache 2.0 on both the model and the upstream licence is the part enterprises will read first, and it is a real difference from the closed diffusion models that came before it.

The 35B number is the point

Diffusion language models have spent two years as a demo with a speed slide. Dream 7B was a research artefact; LLaDA and its successors were open but small or awkward to serve; Inception's Mercury line proved the architecture could hold production voice and coding traffic, but the weights stayed behind an API, and we covered that trade when Inception shipped Mercury 2.5. If you wanted to run a diffusion model yourself at a size that competes with a real coding model, there was nothing to download.

Clover changes the shape of that gap. A 35B model with 3B active parameters is the current sweet spot for self-hosted agent work — big enough to be useful, sparse enough to serve on one GPU, and the exact configuration Qwen, MiniCPM and Gemma 4 all landed on this year. Putting a diffusion decoder on that chassis means the comparison is finally apples to apples: same backbone, same parameter count, same tokenizer family, different generation method. For a plain-English primer on what the two methods actually do differently, AI 101 — What is a diffusion language model? covers the mechanics; the short version is that Clover predicts a block at a time and can look both left and right inside it.

The engineering is in the attention, not the decoder

The interesting technical work here is not "we made it diffuse." It is how DiffuSpace made a linear-attention backbone survive bidirectional decoding.

Qwen3.6's Gated DeltaNet compresses history into a recurrent state — that is how it gets cheap long context — and a causal model reads that state strictly left to right. A diffusion block needs the opposite: every position inside the block attends to every other position, including ones to its right. DiffuSpace's fix is to keep the recurrent state as one more information unit inside the same softmax as the block's tokens, so the model learns at training time how much weight to give compressed history versus the noisy block in front of it. It adds a learnable calibration for block size, because a longer block naturally dilutes attention to history, and a separate zero-initialised convolution branch for right-side context, so the model starts from its pretrained causal behaviour and learns the rightward direction rather than having it imposed.

There is a serving trick in the model card that says as much about the architecture's limits as its strengths: during denoising, Clover can restrict the block to a shared subset of experts covering 90 percent of the routing mass, which cuts how many distinct experts get touched per block. That is a memory-bandwidth optimisation, and the fact that it is needed is a reminder that parallel decoding trades latency for work per token. You are not getting 680 tokens per second for free; you are spending more compute per output token and buying wall-clock time with it.

The benchmark table, read against Qwen's own card

Here is where the launch deserves a slower read.

Clover's 73.6 on SWE-bench Verified beats the 73.4 that Qwen publishes for its own Qwen3.6-35B-A3B — but only just, and the two numbers were not produced the same way. Qwen's card specifies an internal agent scaffold with bash and file-edit tools at temperature 1.0 over a 200K context; DiffuSpace says only that its numbers come from the comparison in the launch blog. A 0.2-point gap between two vendor-run harnesses is noise, and calling it a win over the base model is generous.

The other agent numbers get worse for the claim, not better. Qwen's card puts Qwen3.6-35B-A3B at 51.5 on Terminal-Bench 2.0 — Clover reports 51.7 on Terminal-Bench 2.1, a different version of the benchmark, so the pair cannot be subtracted at all. On τ³, Qwen's card reports 67.2 for the base model against Clover's 65.3. On general knowledge the direction is the same: GPQA Diamond 86.0 for Qwen versus 83.8 for Clover, MMLU-Pro 85.2 versus 84.8. DiffuSpace's "matches or exceeds similarly sized autoregressive models" is true only against its own re-run of the baseline, which it does not publish in full.

None of that makes Clover bad. It makes the honest description "a diffusion decoder that preserves almost all of its base model's agentic coding ability, at roughly parity, with a small general-knowledge cost." That is a genuinely good result for an architecture everyone assumed would lose several points to the conversion. It is not the clean win the headline table implies, and the preview label on the model is doing real work.

The speed claim, against its own baseline

The 680 tokens per second figure is measured on a GB300, BF16, batch size one, on DiffuSpace's internal agent workloads — their data, their prompts, their hardware, a top-of-stack Blackwell part.

The number that matters more is on the older card: 515 tokens per second on a single H200, against 365 for "the DFlash2 setup" on a similarly sized Qwen. DFlash2 is block-diffusion speculative decoding bolted onto an autoregressive model, which is the strongest fair baseline anyone could pick, and against it Clover's advantage is about 1.4 times. The five-to-ten-times figure circulating in Chinese coverage, as TMTPost summarised the launch, compares Clover to a plain autoregressive decoder without speculative acceleration. Both framings are defensible; only one is the comparison a buyer would actually run, and the launch leads with the flattering one.

Raw throughput also stopped being the differentiator some time ago. Mercury 2.5 reports over 1,100 tokens per second in production, and Mercury 3 reports 1,068 with a million-token context. Clover's case has to be that it delivers competitive quality at a speed that a team can host itself, which is a different and narrower pitch than "fastest".

Two claims that do not add up yet

The on-device demo is the one to squint at. DiffuSpace says Clover runs at INT4 on an RTX 5070 Ti with 16 GB of VRAM and hits 101.8 tokens per second on MATH-500, against 48.5 for similarly sized MiniCPM and Qwen comparisons. Thirty-five billion parameters at four bits is roughly 17.5 GB before any KV cache, which does not fit in 16 GB. Either the quantisation is mixed-precision, or experts are being streamed from system memory, or the effective bit width is lower than "INT4" implies. The blog does not say which, and the arithmetic is the first thing a practitioner will check.

The second is in the model card's generation controls. Clover ships with a presence penalty of 0.9, a repetition penalty of 1.05, and two soft biases that ramp up pressure to end the response and to close the thinking block after 4,096 and 8,192 tokens. DiffuSpace is straightforward that these exist because "very long generations can sometimes become repetitive." That is an unusually candid admission for a launch, and it points at the known failure mode of block diffusion: once a block commits, there is no mechanism to revisit it, so a bad commitment propagates. The defaults are a patch over that, not a fix.

What would make this real

Three things, in order of how much they would change the story.

An independent benchmark run. Every number in this release is vendor-run, and the base-model comparison is the vendor's own re-run of a competitor's model. A third party putting Clover and Qwen3.6-35B-A3B through one harness would settle the parity question in an afternoon, and nobody has done it.

The fine-tuning code, which the repository lists as coming soon. A model you can serve but not train is a demo; the moment Clover can be SFT'd on a team's own agent traces, the diffusion decoder becomes a choice rather than a curiosity. DiffuSpace has also promised an agent harness and a desktop coding client, neither of which has shipped.

And usage. The Hugging Face repository shows single-digit downloads in its first day, which means nothing yet — but the gap between "open weights" and "people run it" is where most open releases quietly die. Clover's real test is whether the SGLang path gets used in anger, because that is where numerical differences from the reference implementation will surface.

What to watch

The decision-model variant is the sleeper. DiffuSpace says a Clover-derived model answers up to eight related questions in a single denoising pass, and reports 95.0 percent accuracy against 89.8 percent when the same questions are answered one at a time, with all-answers-correct rising from 62.4 to 80.4 percent on 2,420 linked questions. If that holds up outside their validation set, it is a better argument for diffusion than tokens per second ever was: a decoder that can enforce consistency across related judgements is doing something an autoregressive model structurally cannot. It is also, for now, entirely unverified — and the model is not released.

Diffusion language models have been one benchmark away from mattering for two years. Clover is the first one you can download at a size that competes, and the first whose parity claim is close enough to be worth arguing about. That is progress. It is not yet proof.

Is a diffusion model's speed worth a small quality cost in your stack, or does parity with the base model make it a no-op? Write to us via our contact page.