DFlash 2 doubles Qwen3.8-27B speed in llama.cpp — on one GPU

Local inference had a quietly excellent day: a benchmark that makes an already-fast open model faster, a rescue mission for abandoned AMD hardware, and a single consumer card doing work that used to need a server rack.

A three-day benchmark of DFlash 2's preview build in llama.cpp posted 2.26x decoding gains on Qwen3.8-27B over standard speculative methods — 4.68x with a simple n-gram drafter stacked on top, and up to 8x in specific cases. The numbers come from a LocalLLaMA user who measured against 100 real coding prompts rather than synthetic text, which matters because draft acceptance rates tend to collapse on unpredictable code. It's also the first substantial independent read on DFlash 2 since its parallel-drafting upgrade reached the inference stack — we covered its arrival in DFlash 2 brings 20% faster parallel drafting to the inference stack. The takeaway: speculative decoding remains local AI's best bargain, buying throughput without retraining anything or sacrificing output quality. If these gains survive the merge into mainline llama.cpp, double-speed decoding stops being a benchmark flex and becomes the default experience on a single GPU.

A community-built llama.cpp fork optimized for AMD's long-orphaned GFX906 chips — the Radeon VII and Instinct Mi50/Mi60 — is giving those cards a second life as LLM accelerators. Its author credits GLM with helping build it, and the fork joins a small wave of GFX906-focused projects on GitHub, including a prebuilt package collection that has drawn hundreds of stars. The target is GCN-era hardware that AMD effectively left behind as ROCm moved on to newer architectures. The takeaway: hardware rescue is the quiet upside of the local-AI boom. A 16–32GB card from 2019 running modern quantized models is a genuine option for hobbyists priced out of the current GPU market — as long as they calibrate expectations well below polished, first-party support.

A single RTX 5090 now runs Qwen3.8-27B in heavily compressed NVFP4 format at a true 262K context window in vLLM — reporting 77 tokens per second at short lengths and 64.7 tokens per second at 128K. The LocalLLaMA write-up is a snapshot of how fast quantization and cache techniques have moved: a full-context 27B run was data-center territory not long ago, and it now fits on one consumer card at usable speeds. We covered the model's debut in Qwen open-sources Qwen3.8-27B that runs on consumer GPUs — runs like this explain why it became the default local workhorse. The interesting frontier isn't only bigger models; it's what fits on your desk. When a capable open 27B gets this cheap to serve, private coding assistants and on-prem agents stop needing an API bill at all.

What to watch: whether DFlash 2's preview gains hold once the changes land in mainline llama.cpp — if they do, the benchmarkers' 4x becomes everyone's baseline within a release cycle or two.

Do these numbers change your local setup plans, or are you waiting for the mainline merge before touching anything? Tell us in the comments.

Sources: DFlash support in llama.cpp — PR #22105 · ML-gfx906 builds for AMD GPUs (GitHub) · vLLM (GitHub) · llama.cpp (GitHub)

Houmo moves compute inside memory — and A20 enters production

Beijing chipmaker Houmo used its partner conference this week to lay out two compute-in-memory routes at once — and to say the first chip built on the newer one is already in production. Houmo's A20, its first chip built on a 3D compute-in-memory architecture, has entered production. The part supports up to 16 chips fully interconnected, claims a 50% improvement in compute utilization over the previous generation, and covers models from 3B to 30B parameters, according to co-founder and product

Kling 4.0 goes live with 30-second native video at 10-bit HDR

Kuaishou's Kling AI has spent two years becoming the video model people actually finish projects with. Its new flagship, Kling 4.0, went fully live on October 15 — and this round is less about prettier frames than about the two things that make a generated shot usable in a real edit: length and control. Kling 4.0 generates up to 30 seconds in a single native pass, outputs 10-bit HDR at 4K, 1080p and 21:9, and takes up to 15 reference assets in one task. That budget splits across ten images, fi

Brain organoids classified Braille at 83% — in a trio, not alone

Two stories today: human brain tissue on a chip decoded a tactile signal, and the data-centre company behind some of the largest AI contracts in Europe turns out to be six months behind on its flagship site. Human forebrain organoids grown on a microelectrode array classified Braille letters with 61 percent accuracy on their own and 83 percent when the responses of three organoids were combined, in a peer-reviewed study from the University of Bristol. The team took event-based tactile readings

Suleyman says Microsoft's base model beat OpenAI's on a quarter compute

Microsoft finished pre-training its biggest in-house model yet — then used it to make a point about the company it bankrolls. Microsoft AI completed pre-training of MAI-Base-2, a mixture-of-experts foundation model with 80 billion active and 2.8 trillion total parameters, and says the recipe is 4.2–12.3x more pre-training FLOPs-efficient than DeepSeek-V4-Pro-Base. The run itself is the number worth keeping: 16,000 Nvidia GB200 GPUs for 38 days across August and September, at 21% average model

Read more

Houmo moves compute inside memory — and A20 enters production

Kling 4.0 goes live with 30-second native video at 10-bit HDR

Brain organoids classified Braille at 83% — in a trio, not alone

Suleyman says Microsoft's base model beat OpenAI's on a quarter compute