Pope Leo rebukes Jensen Huang over AI guardrails
The safety argument split in two directions on Monday: a pope taking a named CEO to task over guardrails, and an agent company handing each of its assistants a phone number and a wallet.
Pope Leo XIV, asked aboard the papal flight home from France what the United States and China should do about catastrophic AI risk, rejected the White House's framing and picked a fight with Nvidia's chief executive by name. "I do think that the concerns raised by many of the experts, specialists in AI, should be taken seriously," he told reporters on Monday. "I don't think that that is fake news, as some have said." Then came the specific rebuke: the pope said he had read that morning that the head of Nvidia had announced a way to put guardrails inside certain models, "he's the same one, however, that says there should be no limits placed and no government regulation."
The target was Nvidia's Open Agent Safety Platform, announced that same Monday — the containment stack we wrote up yesterday, where the guardrail moves into a network chip rather than a prompt: Nvidia puts agent containment in a network chip, not a prompt. Huang told CBS News this month that he agreed with President Trump that extinction warnings are overblown. Trump has called the fears a "hoax" and has said he is not putting brakes on the industry; Speaker Mike Johnson told Fox Business on Monday that Congress should not "smother this with red tape" and that "a balance is what we're after."
Why it matters: the timing, not the theology. Trump and Johnson meet AI executives in the East Room at 12:30pm Tuesday — Amodei, Brockman, Pichai, Karp, Zuckerberg and Huang are all expected — and the pope has now put a moral authority figure on the record saying the industry's own guardrails do not substitute for law. Leo's line is not a new mechanism: no treaty, no moratorium, no institute, just a press-conference answer. But it lands on the one argument the labs have been making all year, that voluntary containment is enough, and it lands the day before that argument gets repeated in the White House. The Vatican has been building this position since May, including at the UN last week — The Vatican tells the UN: keep the decision to kill with a human. Nvidia declined to comment.
Separately, Manus shipped version 2.0 on Monday and spun out Cue, a standalone app where each personal agent gets its own email address, phone number, wallet and computer. The rebuild rests on Cascade, an in-house agent harness that loads specialised capabilities only when a task needs them; Manus says that in one tested configuration it used 23.2% fewer tokens, finished 28.2% faster and ran 32% cheaper than its previous system. The rest of the release is a product line: a purchasable Cloud Computer for projects that need to stay running, event-triggered Automations, a desktop Studio with new Video Editor and Game Dev environments, and voice-driven remote control of your own machine. Cue is the interesting half. Each agent can send messages, pay within a budget you set, take a call on your behalf, and work in a group chat with other agents on a shared goal. It is in early access on an invite code.
Manus is independent again after the Meta acquisition was unwound, and this is its first major release since — Manus seeks $4B valuation in first round since Meta unwind. The enterprise version of per-agent identity is the problem Okta and a dozen rivals are trying to standardise right now; Manus has shipped the consumer version, invites and all, where the agent's wallet is the governance boundary.
What to watch: whether Tuesday's White House meeting produces anything binding rather than a balance of adjectives, and whether Nvidia answers the pope's characterisation of its own position.
If an agent holds your phone number and your budget, who is liable when it buys the wrong thing? Tell us in the comments.
