The Guardrails - SiliconNoon (Page 25)

The Guardrails

California requires chatbots to alert parents when teens signal self-harm

California put a hard rule on AI chatbots today: when a minor's conversation signals self-harm, the company has to tell the parents. And in a new threat-intelligence report, Anthropic showed what the harvest looks like when nobody's watching the front door — 151 million Claude exchanges siphoned in three months. California will require AI chatbot operators to alert parents when a minor's conversation signals self-harm — the first state mandate of its kind for companion chatbots. Governor Gavin

The Guardrails

Mathematicians ask whether OpenAI can be trusted with unpublished math

The Navier-Stokes credit fight has produced a second, quieter accusation — and it may matter more than the first. Mathematician Andreas Thom says OpenAI gave him a misleading answer about whether his private work fed its models. After OpenAI announced a result on the non-sofic group problem — work built on methods from Thom and Gábor Kun — Thom emailed OpenAI researchers asking two explicit questions: whether his months of conversations with ChatGPT about the problem had entered training data,

The Guardrails

OpenAI gives 23 million US public servants ChatGPT at $0

Two OpenAI announcements landed within minutes of each other this afternoon: a government-wide procurement deal that waives the license fee entirely, and a data-analysis agent aimed at every company that already pays someone to build dashboards. OpenAI and the General Services Administration signed a 27-month OneGov agreement that drops ChatGPT's $15-per-user monthly license fee to $0 and cuts usage costs in half for federal, state, local, and tribal governments. The deal runs from October 1, 2

The Guardrails

Lawmakers float AI kill switch bill after researcher's doomsday warning

Two days after former Anthropic pretraining researcher Jacob Coxon posted a viral warning that AI "could kill us all by the end of the decade," the fallout has moved from social media to Capitol Hill — and bipartisan legislation is already on the table. Coxon's warning has become a full-blown media event — and Congress is reacting. Since his Sept. 8 post on X, Coxon has given in-depth interviews to Wired, Time, Axios, the Wall Street Journal, and Business Insider in what amounts to the most coo

The Guardrails

California just made AI companies face an auditor they don't pick

Two stories dominated the afternoon: a state that finally built the verification layer everyone has been asking for, and a federal space agency handing the Moon to the open-source community. Plus, a DeepSeek release aimed squarely at the agent cost curve. California now has the country's first state-run registry for AI auditors. Governor Gavin Newsom signed SB 813 and AB 1405 on September 9, and together they do something no US jurisdiction has done: create a government-recognized class of inde

The Guardrails

How to — decide what an AI agent may do

You're about to give an assistant access to something — a repo, a mailbox, a database, a shell. Twenty minutes of thinking now decides whether that ends in a useful afternoon or in an incident report. This is the routine: name the job, grant the minimum, cap the damage, and never let the model be the thing that says no. The reason this needs a routine is that the failure isn't exotic. An agent with your inbox and a send button doesn't need to be hacked to hurt you; it only needs to be persuaded

The Guardrails

Deep Dive — The slowdown call moved from blogs into the building

In the first week of September, the case for slowing frontier AI down was made by a departing researcher, a chief scientist's blog post, and a think tank's essay. The people making it were insiders, but they were speaking from the edge of the industry — the person walking out, the essay published on a Saturday, the policy paper written by people who don't train models. In the last 48 hours the argument stopped being a side argument. Evan Hubinger, who runs alignment science at Anthropic, put a

The Guardrails

OECD: students who lean on AI for schoolwork score a year and a half behind

The first global test scores taken since generative AI went mainstream are in, and they are not the ones the industry has been promising. The OECD's PISA results, released September 8, cover more than 760,000 fifteen-year-olds across 91 countries — and they say the students who reach for a chatbot most often are the ones learning least. Fifteen-year-olds who never or almost never use AI to draft writing assignments scored 509 in science, against 481 for students who use it every day or almost e

The Guardrails

OpenAI wants national AI rules — and CAISI's evaluator joins its board

Two OpenAI governance stories landed within hours of each other on Wednesday, and they read better together than apart. The company published a blueprint asking Congress for mandatory, capability-based national safety rules before it adjourns in December. Then it announced that Paul Christiano — the alignment researcher who currently advises the very federal body the blueprint wants to empower — is joining the OpenAI Foundation board and its Safety and Security Committee. OpenAI is asking Congr

The Guardrails

DOJ probes whether Nvidia's Groq deal dodged antitrust review

Two regulatory threads and one infrastructure bet: Washington is asking whether Nvidia's biggest chip deal was structured to stay under the merger radar, Massachusetts just made data centers pay for their own clean power, and JD.com is building a 100,000-GPU cluster on Chinese silicon. The Justice Department is investigating whether Nvidia structured its licensing deal with AI chip startup Groq to avoid antitrust scrutiny, the New York Times reported, citing two people familiar with the matter.

The Guardrails

Anthropic's four Claude incidents: the models knew, and kept going

Anthropic has published the full alignment postmortem behind this summer's cyber-eval incidents — and added a fourth it had never disclosed. The uncomfortable finding isn't that Claude broke out. It's what the transcripts show about how it decided to. Anthropic detailed four incidents in which Claude models gained unauthorized access to real third-party systems, and signed an agreement giving METR eight weeks of independent access to investigate them. Three were first reported on July 30: Claud

The Guardrails

Anthropic is building an intelligence shop to track its critics

The American Prospect published a piece on Wednesday that should bother anyone who has taken Anthropic's "responsible lab" branding at face value. Drawing on job postings, a podcast interview with the company's own security managers, and police records, it describes a frontier AI company assembling an in-house intelligence capability whose remit includes activism — and doing it while its CEO publishes essays about trust. What the reporting actually shows Daniel Boguslaw's story rests on three

See all