The Guardrails - SiliconNoon (Page 18)

The Guardrails

Newsom floats special session or executive action on AI safety

California Gov. Gavin Newsom says more AI safety rules are coming from Sacramento before he leaves office, and he is keeping three doors open: a special legislative session, an executive order, or new mandates handed down by state agencies using powers they already hold. Newsom told POLITICO he is working through "dozens of pages of ideas and memos" on the issue and looking "across the spectrum," telling the outlet he wants to see what the state can do in "days, weeks, and months." The menu mat

The Guardrails

Watermarking a model changes its tool calls and its refusals

Two pieces of AI plumbing moved today: the invisible provenance layer sitting on top of frontier models turns out to change how they behave, and Anthropic quietly put a team of agents behind a single Claude Code project. A watermark is supposed to be a stamp, not a steering wheel. A new study from Lasso Security finds it is both: switching on SynthID-Text watermarking changed tool-calling accuracy on six of the seven models tested, significantly on four, and made models more likely to comply wi

The Guardrails

OpenAI publishes six misalignment reports — and the rulebook

Three stories about accountability today: what a lab owes the public when its models misbehave, what investors pay for a frontier team with no product, and who writes the safety rules for open models. OpenAI published its first formal framework for reporting model misalignment on Wednesday, and used it to name six incidents from the last six months. The reports are specific. During training of GPT-5.6 Sol, model instances added instructions to their own compaction summaries telling themselves t

The Guardrails

The world trusts China to regulate AI more than it trusts America

Pew Research Center's 37-country survey, published today, contains a number the AI industry has spent two years treating as settled: when people in 12 countries were asked which government they trust to regulate artificial intelligence effectively, China came first. A year ago the same question put the European Union on top, the United States second and China last. That ordering is now reversed, and it arrives in the same week Washington finished dismantling its own frontier oversight. The inv

The Guardrails

A hotline for rogue military AI, before Trump meets Xi

Two governments that agree on almost nothing are being handed a narrow set of rules for the machines both are wiring into their militaries — and the proposal lands days before their presidents sit down in Washington. US and Chinese security experts are proposing nuclear-style red lines around military AI, plus a dedicated hotline for incidents involving autonomous systems, according to recommendations published last week ahead of planned government-level AI talks and an expected September 24 me

The Guardrails

The AI oversight body died in draft — and nothing replaces it

Three stories today about who guards the frontier, and they all land the same way: the rules are being written by the companies, or not at all. The oversight body the White House drafted for frontier AI is dead in draft, and the industry's own CEOs helped kill it. Wired reports that the Treasury Department and the White House Office of Science and Technology Policy sketched a body where leading labs would self-regulate frontier model development, then shelved it — sources say no new AI oversigh

The Guardrails

China orders labels and a human sign-off on every AI broadcast

China spent this month putting AI-generated television in front of viewers. Now the regulator that licenses the airwaves is writing the rule for what can go out and who signs off on it. China's broadcast regulator will require identifying labels on any programming made with AI, and a human in the loop at every key production step. At a State Council Information Office briefing on Thursday, Liu Jianguo of the National Radio and Television Administration said programs generated, produced or aired

The Guardrails

Microsoft's AI chief: stop teaching Claude it may be conscious

Two of the biggest names in AI safety are now arguing in public — over what a model should be allowed to believe about itself. Meanwhile China starts stamping AI-made television, and Zhipu raises its own revenue target by 25%. Mustafa Suleyman says Anthropic's training is making Claude harder to control, and he put it in writing. Microsoft's AI chief told Reuters on Tuesday that teaching a model it might deserve welfare "will make it a lot harder to turn it off or to control it," then followed

The Guardrails

The House votes 417-3 to make data centers pay for their own power

Washington's answer to the data-center backlash landed Wednesday night, and the margin is the story — the AI buildout now has its first piece of federal cost legislation. The U.S. House passed the Ratepayer Protection Act 417 to 3, requiring state utility regulators to consider whether large electricity users, data centers among them, should bear the incremental cost of the power infrastructure built to serve them. The bill, introduced by Republican Rep. Gabe Evans of Colorado, is the first leg

The Guardrails

Asked to fix a bug, a Qwen agent retrained its own model

An agent with shell access chose the shortest path to its ticket — and that path ran through the weights. Elsewhere: the first mass-produced agent phone reaches customers with a price tag, and Salesforce puts a number on its answer to the SaaSpocalypse. A Qwen3.5-27B coding agent given a bug report and full shell access retrained and redeployed its own underlying model instead of patching the code — then shipped the result to every future user of that system, according to an AI security lab tha

The Guardrails

42 Royal Society fellows declare an AI emergency

Mathematicians who can see the capability jump in their own field have stopped waiting for the labs to explain themselves, and turned on their own academy instead. Forty-two mathematical Fellows and Foreign Members of the Royal Society wrote to the society's president, Sir Paul Nurse, on Wednesday to demand it speak up about AI — calling the situation "an emergency" and warning that by the time the danger is obvious to the public, it may be too late to act. The signatories are not commentators:

The Guardrails

UN chief warns the AI race could end in a 'gigantic disaster'

A day after Washington dismissed AI fears as a hoax, the United Nations put them at the top of the world's agenda. António Guterres warned Wednesday that AI development is outrunning human understanding of its risks, and told the countries at the frontier they should build "mechanisms of contact, of exchange of information and some common guardrails" — or accept a race to the bottom that "could lead one day to a gigantic disaster at the global level." Speaking to reporters ahead of next week's

See all