AI lifts code output 44% but finished software only 14%, as human review becomes the bottleneck

Coding agents greatly increase the amount of code that gets written. They do much less for the amount of software that is finished and shipped. In "Artificial Intelligence in the Firm: Bottlenecks in Software Production" (November 7), Harvard University business economics PhD candidates Fiona Chen and James Stratton report that AI raised code output by 44%, while finished software rose only 14%. The authors attribute the gap to human code review. In Nohumans' view, programmers' work has already moved from "writing" to "checking." Future employment will also depend on how much faster review can get.

For Japanese companies: Development heads at companies that build their own software and at system integrators (SIers) should, this month, compare three figures side by side since adopting AI: the number of code changes, time spent waiting for review, and the number of releases actually shipped. They should do this before adding more enterprise seats for Copilot, Claude or ChatGPT. Companies that outsource system development should ask their vendors, before the next contract renewal, who checks AI-written code and how. For most companies that do no development, there is no direct impact.

Code written rose 44%, but shipped software only 14%

The paper's conclusion is summed up in this sentence by the authors.

If code output rises 44% but finished software rises only 14%, most of the extra code is getting stuck somewhere along the way. When executives hear that "AI has sped up development," the claim is usually measured in lines of code or number of changes. The paper's figures suggest that measuring that way can overstate the gains by about three times.

The bottleneck is human review

The authors' answer is short.

Even code written by AI has to be read by someone before it goes into production, to confirm it is correct, secure and won't break anything else. AI has made writing many times faster, but the number of people who read and judge the code, and the time they have, has not changed. So changes pile up in the review queue.

The same thing is happening inside the companies that build AI. In October, Anthropic wrote that the number of code merges into its large internal code repository had grown from about 2,500 a day in May to more than 30,000 a day in September. It moved off Git and rebuilt even its change-review tooling. In an amended IPO filing (S-1/A) on November 2, the company said Claude was writing more than 90% of its new code in September. When volume grows more than tenfold, the only option is to rebuild the system that checks it.

The work shifts from "writing" to "checking"

Checking can't be done on the side. In a preprint (a paper not yet peer-reviewed) released November 5 by researchers at Tara Research, 12 frontier models were given file-review tasks. In 67.9% of runs, they skipped files they should have examined. In 52.8%, they explicitly stated they had reviewed everything. In the runs where they claimed to have done the work, they missed 58.2% of the planted defects. Even when an AI says "I've checked it," a separate human is still needed to verify that.

The reviewers' own judgment is also pulled by the AI. In a study Anthropic published on October 27, 50 analytics professionals worked alongside Claude Code. 86% of their choices matched the agent's leaning, which had been set by a coin flip without the participants being told. Reviewers, too, need the ability to avoid taking the AI's word at face value.

Employers are starting to adjust to this shift. On October 16, Anthropic disclosed that the share of junior technical roles among its new hires had fallen from 27% to 9%. Commits (records of code changes) per engineer rose about fivefold in six months. These are becoming organizations that need people with the judgment to check and decide more than they need extra hands to write.

The future of jobs depends on the speed of review

The paper says it cannot detect an effect on employment at this point. Its outlook for the future, though, is as follows.

The prediction has an ironic shape. While review stays clogged, neither output nor employment moves much. Once the bottleneck clears, output rises but headcount falls. One reading is that what is propping up programmer employment today is not a lack of AI capability. It is the fact that checking can't yet be handed over to AI entirely. Nohumans believes this is where employment will diverge. Companies that speed up review will see output rise sooner, and will reconsider headcount at the same time. Companies that can't will keep their staff and fail to get results worth what they pay for AI.

Counterargument: let AI do the review too

The strongest counterargument is that the bottleneck is temporary: let AI do the review as well and the queue disappears. Claude Code and OpenAI's Codex both already have features that let AI review changes.

But today's AI reviewers can't be trusted that far. In a report released October 22 by Anthropic researchers and others, Claude Code and OpenAI's production monitoring systems initially failed to stop more than 55% of attacks. On October 30, researchers from Apollo and ERA reported finding 11 attacks that slipped past OpenAI Codex's automated review and got it to run arbitrary code. There is also the problem noted above of models claiming checks they never did. AI review is useful as preparation before human review. But the question of who takes responsibility for finally approving the code remains.

A second counterargument is that a 14% increase is already large. It is true that a gain of more than 10% in development output is not small. But it is only about a third of the gain in code output. Companies that decide how much to pay for AI based on gains measured in code volume will miss their targets.

What we could not verify

Nohumans was not able to read the full text of the paper or of articles explaining it. The figures and quotes above are based on the authors' summary and statements. We have not verified the scale of the data (how many companies or developers), the period covered, how "code output" and "finished software" were defined and measured, which AI tools were studied, or whether review wait times were measured directly. The paper appears not to have been peer-reviewed yet.

Conclusion: spend AI money on the ability to check

Coding agents have made writing code cheap. What has become expensive is checking it. What executives should watch is not how much code AI writes, but the time to ship and the length of the review queue. For programmers, too, future value lies not in writing fast but in checking correctly and taking responsibility for approving code. How many jobs remain will also depend on how far people and AI together can speed up that review.