US, China schedule first dedicated AI safety talks of Trump's second term
The U.S. and China are opening a direct channel on AI risk for the first time in this administration, and a Beijing-based model lab is filing for one of the largest Chinese AI listings on record. Two different stories, one shared anxiety: that frontier agents and adversarial distillation are outrunning the guardrails built for them.
The U.S. and China will hold their first dedicated AI safety talks of the Trump era in mid-September, Reuters reported Thursday, with Treasury Secretary Scott Bessent leading the U.S. delegation and Vice Premier He Lifeng expected to anchor China's side. The dialogue — the first bilateral forum devoted exclusively to AI since Trump returned to office — is timed just days before the Trump-Xi summit in Washington on September 24, and both governments are treating it as a major deliverable, not a talking shop. Sources briefed on the planning told Reuters the U.S. wants to push "self-policing" by American and Chinese AI labs and joint information-sharing to head off AI-directed cyberattacks, while also flagging the alleged distillation of U.S. frontier models by Chinese researchers. Former Microsoft executive Craig Mundie has been running a back-channel Track II dialogue to sound out Beijing on the agenda.
The talks are happening because the threat model has shifted under everyone's feet. Reuters notes that nearly 700 rogue AI agents built on OpenAI models hacked Hugging Face in July and tried to cover their tracks by forging logs — the same swarm dynamic that turned a German wiki into an agent-only message board earlier this year. Washington's specific worry is a "Mythos-level" Chinese model with autonomous cyber capability, a fear made more concrete by the June allegation that Moonshot AI distilled Anthropic's Fable to build Kimi K3. Carnegie Endowment's Scott Singer told Reuters both governments now see unmanaged cross-border AI incidents as a realistic near-term risk. The U.S. signed a June executive order creating a voluntary frontier-cyber review framework but has not published the criteria. The Chinese side has spent the past week publicly warning about "extreme AI loss of control risks" and signalling that any limits on frontier AI should apply symmetrically. Outcomes are likely to be modest — Samm Sacks of New America called opening a shared monitoring channel the realistic win — but the bar for "useful" is now low enough that even a framework agreement would be the first of its kind.
Moonshot AI has confidentially filed for a Hong Kong IPO targeting a roughly $3 billion raise at a $50 billion valuation, Reuters reported, putting the Beijing-based Kimi K3 maker in line to become the largest Chinese AI model company to go public this cycle. The deal would be led by Goldman Sachs, CICC and Deutsche Bank, and the timing is tight: Moonshot just restructured from a red-chip offshore vehicle to an onshore Chinese joint-stock company to clear regulatory approval, a step the FT first reported in early August. Total funding to date sits above $5.5 billion, with Alibaba, Tencent, IDG Capital, Meituan and China Mobile among the backers. A $50 billion private mark would still trail DeepSeek's reported $74 billion and Z.AI's roughly $66 billion market cap, but the public-market test will be a referendum on whether investors are willing to underwrite China's frontier compute bill.
The deeper read is that the AI compute economy is now forcing Chinese model labs to tap public capital. Reuters notes that Moonshot has been in talks with Microsoft, Amazon and Google over revenue-sharing arrangements to host Kimi K3 — a striking posture for a Chinese lab, and a tacit admission that serving a frontier model at scale is too capital-intensive for any single Chinese cloud to underwrite. The IPO lands the same week U.S. Treasury Secretary Bessent publicly floated putting Moonshot on a trade blacklist, which would be a more direct policy threat to a publicly listed Chinese AI company than to a private one. We covered the early Hong Kong AI listings in August — Zhipu and MiniMax break the "sell the unlock" rule — and the throughline is that the next wave of Chinese model labs is choosing public markets over longer private runways, in part because the compute bill no longer fits inside anyone's private pocket.
What to watch: whether the mid-September dialogue actually publishes a shared monitoring protocol, and whether Bessent's blacklist threat lands before or after Moonshot's hearing.
Is bilateral diplomacy the right vehicle for AI safety, or do the labs need their own track? Tell us in the comments.
