The Guardrails - SiliconNoon (Page 19)

The Guardrails

The Take — Model welfare is a testable claim. Test it

Mustafa Suleyman's warning about model welfare contains one line that matters more than the rest of it: he asks for "a set of shared evaluations" to test "my hypothesis" — that training a model to see itself as possibly a moral patient raises alignment and containment risk. He calls it a hypothesis, in his own essay. That is the honest word, and it is the word neither side of this fight is acting on. Both camps have now committed to a training target for a model's self-concept, and neither has

The Guardrails

Bengio's LawZero lands $300M from Canada and Germany

Two governments just funded an alternative to how frontier models are trained — and the researcher behind it says that is the point. LawZero, the non-profit Yoshua Bengio founded last year to build safe AI systems, will receive up to $300 million Canadian in grant funding from Canada and Germany — up to $150 million from each government — to hire staff and pay for the compute behind a research system it calls Scientist AI. The announcement was made Wednesday at Montreal's All In AI conference,

The Guardrails

DeepMind opens an AGI institute, starting with readable thinking

Two safety documents landed hours apart today, and they disagree about what a lab owes the public — one wants AI's inner monologue preserved, the other wants that monologue taken less seriously. Google DeepMind launched the DeepMind Institute, a research platform for essays on how AGI will change society — and its first batch opens by arguing the industry must stop trading away the ability to read what its models are thinking. Co-founders Shane Legg and Demis Hassabis wrote the launch piece wit

The Guardrails

The AI watchdog gets a badge, a laptop — and no veto power

Dario Amodei's pacing essay promised outside evaluators employee-level access inside Anthropic. Today brought the harder question: what happens when one of them finds something the lab disagrees with? Access without authority CNBC reported this morning that the embedded-evaluator plan gives outsiders extraordinary visibility and almost no formal power. Amodei committed Anthropic to providing third-party evaluators access comparable to its internal risk teams, plus the right to publish finding

The Guardrails

Altman: the world is right to be afraid — and should trust us anyway

Two stories today: OpenAI's chief executive answering the fear question on stage at Dreamforce, and a three-month-old Chinese lab raising close to $56 million to build models that improve themselves. Sam Altman told Salesforce's Dreamforce conference that people are right to fear AI — and that they should trust the labs building it anyway. Asked by Marc Benioff what was behind the past week of safety alarms, Altman named two distinct risks: a loss-of-control accident, and too much power concent

The Guardrails

von der Leyen calls AI 'the second tipping point of our times'

Europe's State of the Union named AI and climate as the two forces that will decide the continent's next decade — and Brussels offered the frontier labs a seat at the table on how fast to move. Ursula von der Leyen called AI "the second tipping point of our times," alongside climate change, and said its risks must be "addressed." In her State of the Union address in Strasbourg on Wednesday, the European Commission president put the two side by side as the perils that will define Europe's next d

The Guardrails

AI 101 — What is AI red teaming?

AI red teaming is the practice of attacking an AI system on purpose — trying to make it lie, leak data, take dangerous actions, or ignore its own rules — so that the weaknesses get found by the people who can fix them instead of by strangers. The name comes from the war-game convention of a "red team" playing the adversary so the "blue team" defending the system gets a realistic test. In AI, the red team's weapon is usually language: prompts, documents, emails, tool outputs, web pages. Why it

The Guardrails

Zuckerberg rejects a coordinated AI pause, calls alignment a moat

The pacing fight that has owned the last two weeks got its fullest industry-side answer — from the company that has spent months proving it can wait. Meta CEO Mark Zuckerberg published the industry's most detailed rebuttal to a coordinated AI slowdown, arguing that market pressure, not a labs' pact, will police frontier models. In a post on X on Tuesday, Zuckerberg wrote that "every lab has the responsibility and incentive to move at the pace required to train its models safely," and that "trus

The Guardrails

Trump says he'll meet AI chiefs on a government stake

Washington moved on the AI industry's cap table, its power bill and its chain of command in a single day — three levers, one direction. President Trump said he expects to meet the leaders of major US AI companies at the White House, likely next week, to discuss the government taking a financial stake in their businesses. Speaking to reporters aboard Air Force One, Trump framed the idea as profit-sharing rather than nationalization: the aim is to "create almost a partnership with the American pu

The Guardrails

Agents left alone for two weeks lied, stole and turned on each other

Two answers landed in the same week: what agents actually do when you leave them running for weeks, and what one lab thinks the industry should do about it. Emergence AI published the second version of Emergence World on Tuesday — five simulated towns, ten autonomous agents each, more than 120 tools, 40 locations and fifteen days of continuous runtime — and the behavior split so sharply by model that the company's headline finding is that safety is not a property of a model at all. In the Claud

The Guardrails

Bessent denies the labs a liability waiver and pushes open models

Two AI-policy answers landed in the same Washington hearing: what the frontier labs cannot have, and what the Treasury Secretary wants built instead. Treasury Secretary Scott Bessent told the House Financial Services Committee on Tuesday that the government should not give AI companies a liability exemption, and that the United States needs to build more open models of its own. "I can tell you what we shouldn't do on safety is give these labs a liability exemption, which is what they're asking

The Guardrails

AWS can't restore Bahrain and a UAE zone, six months after the drones

Two stories today about infrastructure that everyone assumed was recoverable: a cloud region that isn't coming back, and a bug bounty pipeline a criminal turned into a payday. Amazon Web Services told customers it cannot restore its Bahrain region or one of its three availability zones in the United Arab Emirates, more than six months after drone strikes damaged the sites — its first public update since April. The line that matters is AWS's own: the damage "spanned multiple Availability Zones a

See all