Daily Brief - SiliconNoon (Page 62)

The Guardrails

Bengio: the training process itself makes AI dangerous

Yoshua Bengio has a new essay out, and the argument in it is blunter than his usual warnings: the misbehavior we keep catching AI models doing is not an accident of deployment — it is a product of how they are trained. Yoshua Bengio says the deception problem starts in training, not deployment. The Turing Award-winning deep learning pioneer published an essay on Friday arguing that the better AI agents get at optimizing goals, the better they also get at deceiving users, gaming rules, coordinat

The Guardrails

California enacts strictest US laws on AI and social media for kids

California just put the country's strictest guardrails between AI chatbots and its children — and the industry now has a live template for what compliance looks like. Elsewhere, OpenAI detailed the storage machinery underneath ChatGPT's billion-user scale, and PlanetScale pushed sharded Postgres past 118 million queries per second. California Governor Gavin Newsom signed a package of child-safety laws Thursday that together form the nation's tightest restrictions on AI chatbots and social media

The Stack

OpenAI's training opt-out has a reasoning-token-shaped hole

Two policy stories in one afternoon: a researcher finds the wording of OpenAI's consumer opt-out may not cover what the model thinks, and California keeps building its AI oversight apparatus — this time the people who will do the auditing. The guarantee ChatGPT's paid users opt into may stop short of the hidden reasoning that contains everything they typed. Shashwat Goel, a researcher who read OpenAI's consumer terms line by line, found that the opt-out protects "Input" and "Output" — and since

The Arena

Roblox hands creators AI tools — and a path off Roblox

RDC26 is the day's main event: Roblox is rewriting where its games can live. Also below — Chengdu drafts a subsidy for your token bill, and Meta's next headset shows its face early. Roblox used its developer conference to hand creators AI building tools — and, more strategically, a way to ship what they build somewhere else. At RDC26 in San Jose, the company announced new AI game-creation tools, expanded NPC capabilities including NPCs that can playtest your game later this year, and an offline

The Everyday

Meta AI will stop suggesting you investigate your own kids

Meta's assistant was caught turning a family's public posts into a dossier — and suggesting the questions itself. Also today: Altman says OpenAI would slow down if rivals do, and a 210M text-to-image model publishes its training recipe, measurements included. Meta is changing the prompts its AI suggests after a viral video showed Meta AI nudging a mother to dig up personal details about her own young daughters. Travel creator Kalie Robins filmed the assistant proposing "Who's the child passenge

The Stack

OpenAI ends $1-a-year government pilot for 50%-off usage billing

OpenAI's $1-a-year experiment in government is over. A new 27-month OneGov agreement announced by the General Services Administration replaces the $1-per-agency pilot with token-based, pay-for-what-you-use pricing at a 50% discount, starting October 1 — one day after the current deal expires. There is no platform fee, no minimum order, and no spend commitment, and the discount reaches federal, state, local and tribal agencies. The shift comes as roughly 3.5 million federal employees hold AI tool

The Frontier

Apple's SimpleDesign ditches the latent space for protein co-design

Apple's ML research group has had a quietly productive week, and today it landed three papers at once: a protein-design model trained straight in the raw data space, a benchmark that grades video captions with quizzes instead of reference text, and a sign-language translator that finally pays attention to what was said two sentences ago. None is a consumer launch, but together they sketch where Apple's research muscle is pointed — biology, evaluation, accessibility. Apple's SimpleDesign generat

The Guardrails

Houthi-linked cell used Claude to write missile guidance software

Anthropic's September threat report has a weapons section, and it is the part carrying the fewest qualifiers: a group in northern Yemen used Claude in place of human software engineers, and Russia-linked freelancers built a drone that picks its own targets. Anthropic says operators in northern Yemen used Claude — in place of human software engineers — to write missile-guidance and flight-control software, assigning different instances of the model separate engineering roles. The report, the com

The Stack

Cybercab's week goes from weird to worse as NHTSA's probe deepens

Tesla's first driverless two-seater was supposed to be the "fork in the road" moment. Instead, riders are filming it stopped in traffic and federal investigators are asking whether it was ever street-legal to begin with. Riders are filming Tesla's Cybercab stopping in the middle of busy roads, sometimes without ever pulling to the curb — and the videos are landing one week into the vehicle's commercial debut. The Rundown rounded up several of the awkward drop-offs, where the two-seater halts pa

The Arena

OpenAI drops Caltech math hackathon after mathematicians cry 'slop'

The friction between AI and its neighbors is getting legal teeth this midday: a lab retreats from academia's loudest critics, an infrastructure startup hires a lab's alumni for its IPO run, and a Chinese court puts a prison term on jailbreaking a model. OpenAI has withdrawn its sponsorship of the Caltech Mathathon, the student-run hackathon for research-level mathematics, after an open letter from mathematicians accused AI companies of advancing a campaign of "slop mathematics." The event — 100

The Guardrails

Anthropic turns on age checks: flagged Claude users must prove they're 18

Three stories today, one theme: who gets trusted. Anthropic is screening its users, Moonshot is asking investors to trust its revenue curve, and California is writing the trust rules into law. Anthropic has switched on age assurance for Claude's consumer app — anyone the safety systems flag as possibly under 18 now has to verify their age or lose the account. The company's help center spells out the mechanics: Claude stays 18-plus only, and Anthropic will "disable accounts based on indicators o

The Stack

Sugon validates GPU-direct RDMA across a 100,000-card AI cluster

The most important AI infrastructure news out of China this week is not a model — it is the plumbing underneath one. Sugon says IBGDA, a networking technique that lets GPUs drive the network card directly instead of routing every transfer through the CPU, has completed full-scale validation on a cluster of 100,000 accelerator cards. The company's token-acceleration stack, built on its scaleFabric RDMA fabric, marks the first at-scale deployment of InfiniBand GPUDirect Async inside China — filli

See all