Coreset pruning cuts 80% of VLM tokens, keeps accuracy intact
Vision-language models are drowning in their own pixels, and a Chinese Academy of Sciences team has a training-free fix: cut up to 80% of the visual tokens a multimodal model has to chew through — and lose nothing.
The CAS Institute of Automation's Zidong Taichu team has released GMC (Grounded Message Coreset Pruning), a training-free method that cuts up to 80.2% of a vision-language model's visual tokens while retaining 97.78% of full-model capability — and in its most aggressive configuration, actually scores slightly above the uncompressed model. The method, described in the paper "Messages, Not Tokens: Grounded Coresets for Faithful VLM Compression" (arXiv:2608.02134), works out of the box on Qwen2.5-VL-7B and LLaVA-1.5-7B, with no fine-tuning, no auxiliary models, and no OCR or detection tools.
The problem GMC attacks is the quiet cost driver of multimodal inference. High-resolution images become hundreds to thousands of visual tokens, and every token rides through the decoder and occupies the prompt KV cache — inflating memory, slowing generation, and raising deployment cost. The standard fix, Top-K selection, scores each token independently and keeps the best ones. That's the flaw: high-scoring tokens cluster on an image's salient region, repeatedly covering the same evidence, while scattered details — text, numbers, chart axes, spatial relationships — get dropped, and whatever the discarded tokens carried is gone for good. Hence the industry's familiar "compress and lose accuracy" tradeoff, plus extra hallucinations.
GMC's shift is conceptual: treat the visual sequence as a collective message the model consumes, not a bag of independent tokens. Its selection stage builds evidence from three angles — question-relevant regions (found through the model's own vision-text attention), visual appearance structure, and spatial coordinates — then chooses tokens by their marginal contribution to evidence not yet covered. A second "population transport" stage folds the hidden states of discarded tokens into the selected representatives before the sequence is physically compacted, so no information is silently deleted.
The numbers hold across settings. On Qwen2.5-VL-7B, keeping just 256 of 1,296 visual tokens retains 97.78% of average capability, and the more aggressive L16 configuration lands at 100.36%. On LLaVA-1.5-7B the method transfers cleanly, holding roughly 99.8% of average performance with as few as 64 tokens. In a long-document QA test with nearly 16,000 visual tokens, GMC preserved 98.87% of answer quality while cutting prompt KV cache by 73.94% and speeding up end-to-end inference 1.26x. It also held up better on hallucination benchmarks like POPE and HallusionBench than equal-budget baselines.
Why it matters: visual token redundancy is one of the biggest remaining levers on inference economics, and a training-free, drop-in compressor that keeps accuracy is the kind of result inference stacks can adopt immediately — no retraining budget required. It's also a neat reframing of the whole problem: compression isn't about deleting tokens, it's about deciding which tokens carry the message and letting the rest fold into them. Papers like this are how the efficiency gains that eventually show up in API prices actually get made.
What to watch: whether the team open-sources the GMC code, and whether the method holds up on larger, reasoning-heavy VLMs — the test that separates efficiency tricks from real wins.
How much accuracy would you trade for a 5x cut in vision inference cost? Tell us in the comments.
