Anthropic's AI safety bill needs a judge — and a $1B threshold
The most consequential AI document of the week is not a paper or a model card — it is a bill, and the company that wants it passed is the one it regulates. Anthropic's representatives have been quietly pitching a draft AI safety and security law to Capitol Hill lawmakers, and Politico obtained the confidential text on October 16, with three people familiar with the efforts confirming the outreach. An Anthropic spokesperson did not dispute it, telling Politico the draft "represents the proposals outlined in Anthropic's Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June."
Strip the legislative language and the core is a number. The government could block the release or use of an advanced model only if it poses a foreseeable, imminent risk of more than $1 billion in damages, or of killing or seriously injuring more than 50 people. The risk must also come from exactly one of three places: the creation or release of a biological weapon, the model evading its developer's controls, or a cyberattack. Anything else — a slow-rolling labor shock, a flood of fraud, a system that quietly degrades a public service — is not a reason Washington may say no.
The threshold is the tell
Every incident the frontier labs disclosed this summer — agents probing and hacking websites and platforms — falls below that line, a point Politico's reporting makes plainly: none reported so far appears to have caused widespread deaths or over $1 billion in damages. That is the design, not an accident. This summer's disclosures were embarrassing but small, and Anthropic is proposing to close the window while the numbers are still small. The bill's risk language mirrors California's SB 53, signed into law last year, which is either a courtesy to the most ambitious state regime or a way of making the federal text a familiar default.
The draft is not only a shield. It would require frontier developers to keep safety frameworks, publish transparency reports for new models, maintain whistleblower protections, and report critical incidents directly to the Commerce Department. Independent evaluators, registered with the Commerce Department, would periodically assess labs and their models — labs may pay them, as long as payment is not tied to the outcome. An evaluator that finds an "imminent enumerated risk" must report it to Commerce within 72 hours. This half of the bill is close to what safety advocates have asked for. The other half is what they are objecting to.
Nobody but a judge gets the switch
Commerce could not unilaterally stop a model even after concluding it was dangerous. Under the draft, the department would have to ask a federal judge in the District of Columbia to order a suspension — and the text explicitly bars any other federal department, agency or officer from suspending a model's development, deployment, or internal use. The draft would also pre-empt state and local rules on public risk disclosures, third-party audits and critical-incident reporting, while leaving intact the rest of the state safety stack.
That pre-emption clause is where Anthropic's June position collides with its October one. In June, as states legislated and Congress stalled, Anthropic argued that "absent a strong federal law, states should retain full authority to legislate," per Reuters. Having now drafted what counts as a strong federal law, the company proposes federal ceilings on exactly the domains where California, Illinois and New York have been writing rules. Its own head of state and local government relations previously told Politico that Anthropic hopes states keep passing ever-stronger laws — which is a coherent position with this draft, and a contradictory one without it.
Why the lab is writing law
No other frontier lab has put actual legislative text on the table, and the reason is Anthropic's year. The Trump administration banned Anthropic from the Pentagon in February; in June, Washington forced the company to suspend its most advanced models for nearly three weeks; and CEO Dario Amodei spent September publicly warning that a swarm of AI agents could take control of "the entire internet" within six to 12 months, Reuters reports. A lab that can be shut down by administrative pressure with no hearing, no threshold and no appeal will eventually try to write the hearing into law.
The winners are legible: frontier labs get a bright-line federal threshold, a single judicial forum, and pre-emption of the most aggressive state regimes — predictability with a floor so high it has never been met. The losers are state regulators, and the executive branch itself, which under this text could not act against a model its own agencies considered an imminent risk without a judge's signature. Between them sits everyone downstream of the three enumerated risks: the bill makes "the government stopped a model" a scenario that can only follow a bio, cyber or control-evasion event with a billion-dollar body count attached.
The pushback — and what moves next
The criticism splits three ways. Charlie Bullock of the Institute for Law & AI called the proposal "pretty reasonable" and "more than other frontier labs have done," while judging it weaker than the FRONTIER Act, the House bill from Representatives Obernolte and Trahan introduced this summer. That is roughly the good-faith verdict: credit for text, marks off for the ceiling.
The harder objections go to scope. Former Pentagon official Matt Spence wrote on X that the draft "actually precludes action against pretty much any unforeseen X-risk, narrowing government authority to only cyber and bio." Attorney Jay Shooster called it "outrageous and unacceptable," and journalist Kelsey Piper's read is that the lab's real fear is an executive branch that can flexibly make them stop. The structural critique is the strongest one: a dollar figure and a casualty count are ex-ante numbers, and the risks labs themselves describe as their reason for existing — gradual, cumulative, novel — are the ones no threshold catches.
What to watch: Congress is out of session until after next month's midterm elections, so this is positioning text, not a markup. When lawmakers return, the test is whether the FRONTIER Act absorbs Anthropic's evaluator machinery while keeping a lower bar — the legislative route mapped in Senate weighs a duty-of-care law that could block unsafe AI models — and whether Democrats hold the line for state authority, the exact fight the authors of State AI laws' authors ask frontier labs to slow down already framed. Commerce did not respond to Reuters' request for comment. The next Anthropic framework update, whenever it lands, will tell you which part of this draft the company thinks it can win.
Is a billion-dollar threshold a responsible bright line, or the safest possible way to write a law that never fires? Tell us in the comments.
