Anthropic's draft AI safety bill reportedly leaves model shutdowns to a single federal judge and overrides state laws, drawing an "unacceptable" verdict from safety advocates

Anthropic, known for calling for AI regulation, is pitching a draft AI safety bill it wrote itself to lawmakers, Politico reported on Oct. 16 (the report was carried by US News, Bloomberg, Yahoo and The Star). The reported draft has two core provisions. Only the federal district court in Washington, D.C. would be able to order a dangerous model shut down, and the federal law would override state laws. This article is based on news reports and leaks. We have not seen the text of the draft or any official explanation from Anthropic, so we can describe the details of the bill only as far as they have been reported.

Even so, if the reports are accurate, the picture is clear. In this draft, the "regulation" Anthropic has called for would take the power to shut down models away from the executive branch and the states.

Only a D.C. court could order a shutdown

According to the reports, the draft puts the power to order a model shutdown in a single court, the federal district court in D.C. In other words, regulators could not shut down a model on their own judgment, and states could not shut one down under their own laws. Anyone seeking a shutdown would have to file suit in D.C. and persuade a judge.

The draft is easier to understand against the events of the past three weeks. On Oct. 10, Microsoft's Nadella called for an "emergency brake." He wrote that authorized people should be able to stop a model in the middle of a task, and that controls should sit "outside the model." Brad Smith also told the BBC that incident reporting should be mandatory and that an "emergency brake" should be written into law. The draft does include a brake. But if the reports are accurate, only one court would be able to pull it. With AI agent incidents, the main dispute has been how long it takes to go from detection to response. In Anthropic's own Philadelphia police case, it took 72 days to notice a false report, and the city called the delay in reporting "unacceptable." A design that runs shutdowns through litigation runs directly against those demands for speed.

Overriding state laws would wipe out existing rules

The draft's other main provision would override state laws. In the U.S., states have moved first while Congress has failed to pass a comprehensive law on frontier AI. Some state laws, such as California's SB 53, already require frontier model developers to publish their safety frameworks and report incidents. The reports do not say which state laws the draft would cover. But if a federal law overrides state regulation, the states could lose those powers as well.

Overriding state laws has until now been something the industry argued for in opposing regulation. Here, a lab that presents itself as safety-focused has written it into its own bill and brought it to lawmakers. This is part of why reports described the pitch as made "quietly." What Anthropic says in public in favor of regulation and what the text it gives lawmakers would do point in different directions.

"Unacceptable," say safety advocates

The strongest reaction came from people who take AI risk seriously, not from opponents of regulation. Zvi Mowshowitz, a widely read writer on AI risk, gave his post on the leaked draft the following title:

The criticism comes from the camp that would normally be closest to Anthropic. The critics' main point is that the draft is weaker than current law. By limiting shutdown authority to one court and overriding state regulation, the bill could look like federal safety regulation while actually leaving less enforceable authority than exists today. The critics argue that the idea that "something is better than no regulation" does not hold when state laws are already on the books.

The policy landscape

The timing of the draft also matters. The White House Office of Science and Technology Policy (OSTP) is expected to produce a legislative proposal in late November, and the FTC has issued civil investigative demands to frontier labs. This fall, the shape of a federal framework is being decided. A lab that gets its own draft to lawmakers first can shape where that framework starts.

On Oct. 15, Anthropic announced an arrangement giving METR access "roughly equivalent" to that of its internal risk evaluation team, with no veto over publication. The same week brought both an announcement that makes the monitored company more transparent and a draft that would narrow the powers of those who monitor it.

What remains unknown

The draft has not been made public, and everything described here is based on news reports. Much remains unclear. Which models, by size, would be covered? Who could bring a case in the D.C. court: only the Justice Department, or state attorneys general as well? Would the override cover only safety regulation, or AI law in general? Is this Anthropic's final position or a starting point for discussion? And will Anthropic respond publicly to the reports and the criticism?

Still, whatever the details, the two reported provisions both point the same way: putting shutdown authority in a single federal court and overriding state laws. To the question of who can shut down a model, the draft answers: not the executive branch, not the states, but a judge in D.C. A bill presented as a safety measure has been judged weaker than current law by the people who care most about safety.