DFlash 2 doubles Qwen3.8-27B speed in llama.cpp — on one GPU
Local inference had a quietly excellent day: a benchmark that makes an already-fast open model faster, a rescue mission for abandoned AMD hardware, and a single consumer card doing work that used to need a server rack.
A three-day benchmark of DFlash 2's preview build in llama.cpp posted 2.26x decoding gains on Qwen3.8-27B over standard speculative methods — 4.68x with a simple n-gram drafter stacked on top, and up to 8x in specific cases. The numbers come from a LocalLLaMA user who measured against 100 real coding prompts rather than synthetic text, which matters because draft acceptance rates tend to collapse on unpredictable code. It's also the first substantial independent read on DFlash 2 since its parallel-drafting upgrade reached the inference stack — we covered its arrival in DFlash 2 brings 20% faster parallel drafting to the inference stack. The takeaway: speculative decoding remains local AI's best bargain, buying throughput without retraining anything or sacrificing output quality. If these gains survive the merge into mainline llama.cpp, double-speed decoding stops being a benchmark flex and becomes the default experience on a single GPU.
A community-built llama.cpp fork optimized for AMD's long-orphaned GFX906 chips — the Radeon VII and Instinct Mi50/Mi60 — is giving those cards a second life as LLM accelerators. Its author credits GLM with helping build it, and the fork joins a small wave of GFX906-focused projects on GitHub, including a prebuilt package collection that has drawn hundreds of stars. The target is GCN-era hardware that AMD effectively left behind as ROCm moved on to newer architectures. The takeaway: hardware rescue is the quiet upside of the local-AI boom. A 16–32GB card from 2019 running modern quantized models is a genuine option for hobbyists priced out of the current GPU market — as long as they calibrate expectations well below polished, first-party support.
A single RTX 5090 now runs Qwen3.8-27B in heavily compressed NVFP4 format at a true 262K context window in vLLM — reporting 77 tokens per second at short lengths and 64.7 tokens per second at 128K. The LocalLLaMA write-up is a snapshot of how fast quantization and cache techniques have moved: a full-context 27B run was data-center territory not long ago, and it now fits on one consumer card at usable speeds. We covered the model's debut in Qwen open-sources Qwen3.8-27B that runs on consumer GPUs — runs like this explain why it became the default local workhorse. The interesting frontier isn't only bigger models; it's what fits on your desk. When a capable open 27B gets this cheap to serve, private coding assistants and on-prem agents stop needing an API bill at all.
What to watch: whether DFlash 2's preview gains hold once the changes land in mainline llama.cpp — if they do, the benchmarkers' 4x becomes everyone's baseline within a release cycle or two.
Do these numbers change your local setup plans, or are you waiting for the mainline merge before touching anything? Tell us in the comments.
Sources: DFlash support in llama.cpp — PR #22105 · ML-gfx906 builds for AMD GPUs (GitHub) · vLLM (GitHub) · llama.cpp (GitHub)
