DeepMind alumni's Inherent says its Faraday agent beats GPT-5.5 at reproducing research
A London lab founded by Google DeepMind alumni says its new agent, Faraday, outperformed Claude Opus 4.8 and GPT-5.5 at independently reproducing the findings of published scientific papers — and it did it on a 27B-parameter Qwen 3.6 model, a fraction of the frontier systems' size. Inherent emerged from stealth just weeks ago with a $50 million seed round, and this benchmark claim is its first real proof point.
The company is upfront that the task was narrow: replication, not discovery. But that's the right training ground — reproducing a paper without knowing the answer in advance demands reading methods, forming hypotheses, designing experiments and interpreting results, which is roughly how human PhD students cut their teeth. Chief scientist Edward Hughes told TechCrunch the interesting part was not beating frontier agents but how they got there: rather than training primarily on studies of how science is conducted, Inherent leans on reinforcement learning, rewarding good experimental outcomes so the behavior generalizes toward its actual goal — agents that can discover new knowledge, not just verify old results.
The claim deserves skepticism until independent runs confirm it, since it comes from the company itself. Still, the shape of the result matters more than the score: if research taste can be taught to small models through rewards instead of scale, the moat around frontier labs gets thinner. We covered a related signal earlier today — Mystery model Ox Alpha tops GPT-5.6 — where a smaller challenger again embarrassed bigger systems.
Your local LLM isn't dumber than the API version — your stack is quietly sabotaging it. A viral Level1Techs forum post (now on the Hacker News front page) ran controlled experiments on Qwen3.6-27B showing that changing only the attention backend inside an inference engine — same GPU, same weights, same prompt — flips enough next-token probabilities to derail tool calls entirely: one run dialed the wrong Cisco interface, then executed the wrong command twice trying to recover.
The rot compounds as context grows. With an int4 KV cache the model's "IQ drops like a rock" past roughly 40k tokens and a recoverable mistake becomes unrecoverable; across five weight formats, Nvidia's own NVFP4 release came dead last with about half of sampled tokens flipped by 88k of context, while a community INT8 build beat the official FP8 checkpoint. The author even found tensor parallelism flipping results — two GPUs failed a call that one and four GPUs passed.
The takeaway for anyone running local inference: treat quantization and runtime configuration as accuracy decisions, not just speed knobs, and benchmark your own stack on long, tool-heavy workloads — not three zero-shot prompts. Labs publishing "impossibly low" divergence numbers for quants without methodology disclosure deserve the side-eye too.
What to watch: whether anyone replicates Inherent's Faraday numbers independently — and whether the Level1Techs team ships its promised distributable testing rig so you can measure your own stack's drift.
Running local models for real work? Have your own token-flip horror stories — or a quant you swear by? Tell us in the comments.
