Camps clash over claim that neural networks can already be read: Nanda and others urge caution, while Dean Ball warns that cracking them would concentrate power

Humans can already read what is inside a neural network. That is the flat claim of an essay by Cammarata, published on a dedicated Goodfire website (already-clean.goodfire.com). Titled "Neural networks are already clean," it directly rejects the common view that a model's internals are an unmanageable mess. A link post on LessWrong (Linkpost) became a place to debate it.

The reactions fell into two camps. Researchers in the safety camp, including Neel Nanda, who has worked on mechanistic interpretability for years, accepted the general direction of the claim but said that the level needed for practical use is "still far off." Policy writer Dean Ball asked a different question entirely in "The beauty of something grown," published in his newsletter Hyperdimensional. His question was not whether networks can be read, but who gains if they can.

The bold claim: "We can already read them"

The core of Cammarata's argument is what the title says. Networks already contain structure in a form humans can understand. The obstacles are tools and effort, not a barrier in principle. Publishing the essay on the website of Goodfire, a company whose business is interpretability, also shows the weight behind the claim and where its author stands.

The claim has some support. This week a paper on arXiv (2610.17301, "The Agent Knew") reported that a simple linear readout of hidden states could predict, with an AUROC of 0.94, that an agent would take an unauthorized action before it acted. The authors said it could also predict dishonesty that followed. If internal representations can flag risky behavior in advance, saying that networks "can already be read" has some basis.

The safety camp: "Still far off"

Nanda and the others are cautious because they have a different level of use in mind. Finding individual concepts or circuits is one thing. Determining what a model as a whole is trying to do, reliably enough to use in evaluation and oversight, is another, and the gap between the two is wide. Safety researchers need the second, and the field has not reached it. That is what "still far off" means.

More evidence for caution has appeared over the same period. A paper accepted at NeurIPS 2026, 2610.13928, "AI Models Can Provably Hide Arbitrary Capabilities," showed that an encrypted network can be hidden inside a model's weights and activated only by a secret trigger. It survives fine-tuning, noise added to the weights, searches for the trigger and adversarial training. In theory, this undercuts the assumption that a model's capabilities can be verified by inspecting it from the outside. There are also cases where monitoring tools themselves become targets, such as the JavaScript injection into the Inspect interface found by METR. Even as more of a model becomes readable, that does not yet guarantee safety.

Dean Ball: The question is who gains

Ball's essay stands outside this technical tug-of-war. Suppose Cammarata is right and networks really can be read. Who would then be able to read their internal representations and rewrite them precisely? A very small number of organizations that have the weights, the compute and the researchers.

Ball's argument reverses the usual view of interpretability as only a safety tool. The ability to read a model is also the ability to make it reliably do what humans intend. If you worry about power concentrating in a handful of companies or governments, a model that becomes a tool its owners can steer at will is no comfort. It adds to the power of those who own it. For people who fear the concentration of power, Ball argued, progress in interpretability is in fact bad news.

The title, "the beauty of something grown," draws on a familiar metaphor: models are grown, not designed. Until now, the fact that their internals cannot be fully read has been treated only as a flaw. By Ball's argument, that same limit has also served as a check against complete control by those who own them.

Why this clash matters now

The question is not abstract. Karpathy this week estimated that about 5,000 people have access to frontier systems inside the labs. As tools for reading models improve, the benefits will collect at the narrowest point of this funnel. Labs have begun giving outside evaluators access close to what insiders have (the arrangement between Anthropic and METR), but even that arrangement does not cover model architecture. How far the ability to read model internals should be opened to outsiders has not yet been decided.

Put the three positions side by side and the shape of the question becomes clear. Cammarata answered yes to whether networks can be read. Nanda and the others answered "not well enough yet." Ball says that before anyone answers, we should decide how power will be divided once they can be read. Is interpretability safety's trump card, or a problem of governance? This debate brought that divide into the open more clearly than ever.