Opinion · Oct 26, 2026
Merchant calls Anthropic's AEGIS bill "regulatory capture"
Safety advocates say it is "too weak on catastrophic risk," while ethics critics say "it is illegitimate for a lab to write the rules at all." Their reasons point in opposite directions, but both reach the same verdict: it is a bad bill
Koji Yamamoto · Economics Analyst
Key points
- Writing in Blood in the Machine on October 25, Merchant opposed the AEGIS Act and called it "regulatory capture." His piece is titled "We cannot under any circumstances let AI firms govern the future"
- The safety camp criticizes the bill as too weak on catastrophic risk. Merchant's reasoning runs the opposite way: he argues that a process in which the lab writes its own rules lacks legitimacy
- Two camps that have long clashed over AI regulation now oppose the same bill for separate reasons, leaving Anthropic with that much less support as it pushes the bill
Brian Merchant, a writer on the ethics-critique side of the AI debate, has come out against Anthropic's AEGIS Act, calling it "regulatory capture." He made the case on October 25 in his newsletter, Blood in the Machine. TokenPost also reported on the critique.
The bill had already drawn criticism from the safety camp, which argues that it is too weak on catastrophic risk. Merchant's objection points the other way. Before asking whether the bill's provisions are strong or weak, he says, it is illegitimate for the company being regulated to write its own rules at all. The two camps start from opposite reasons but arrive at the same conclusion: it is a bad bill.
What Merchant said
The title of his piece states his argument directly.
We cannot under any circumstances let AI firms govern the future
This is not an objection that fixing the bill's text would resolve. It goes to the starting point: who writes the rules. "Regulatory capture" usually means a regulator coming to serve the industry it is supposed to oversee. Merchant uses the term for a situation in which the industry writes the regulation's first draft before it has even gotten inside the agency. If the frontier labs that are subject to regulation set the bill's framework, he argues, the bill becomes an official stamp of approval on practices the labs have already chosen.
Merchant has long criticized what technology does to workers and to society. He has clashed on many issues with the safety camp, which focuses mainly on catastrophic risk from AI. Now he is opposing a bill from a lab that puts safety front and center.
Two critiques running in opposite directions
The safety camp measures the bill by whether it is enough. In this view, the obligations it imposes have not kept up with capabilities that could lead to catastrophe. The critique therefore pushes for stricter thresholds and stronger obligations.
Merchant measures it by who decides. In that view, stronger obligations do not solve the problem. Raising the strictness inside a framework the lab wrote leaves the power to write the rules with the company. Taken to the extreme, the more the text is revised the way the safety camp wants, the more firmly the lab's blueprint is locked in, as Merchant sees it.
The two critiques also call for different fixes. The safety camp asks for "stronger," while Merchant asks to "take it out of the labs' hands." They still agree on one point: the bill as it stands should not pass.
The legitimacy question extends to Anthropic's other moves
The pattern of a lab writing its own rules is not limited to this bill. The same shape shows up in other recent moves by Anthropic.
According to NBC, Anthropic brought in legal scholars and democracy experts to discuss who should decide Claude's values. However, Anthropic reportedly keeps the final say. The same pattern appears in how it treats outside evaluators. Reuters reported that Anthropic and OpenAI had sought strong editorial control, through contracts, over what third-party evaluators embedded in their companies could publish. Anthropic has called that report inaccurate. On October 15, it published, as a primary source, an arrangement stating that Anthropic has no veto over what METR publishes.
In each case, Anthropic builds a way to hear outside voices. Still, Anthropic is the one that draws the final line. Merchant's critique can be read as an objection to extending that pattern into law.
What to watch next
Anthropic now has less support for the bill. People who put safety first call it "weak," and people wary of the labs' power say "its origins are illegitimate." Strengthening the text would answer the first group but not the second. The only way to answer the second is to change the process that decides who writes the text.
Much about the bill is still undecided: who in Congress will sponsor it, how the language preempting state law will be rewritten, and how the thresholds that determine what it covers will be revised. In the safety camp, Zvi Mowshowitz has said he will publish a piece about Anthropic. How these questions address the legitimacy of the process will decide whether the opposition from both camps continues.
Editorial cartoon
