Coding Agents Produce More Code, Not More Software (2026)
When a company rolls out coding agents, the first dashboard that lights up is usually code volume. Lines, commits, and pull requests all go up. Intuition follows: if we write faster, we should ship faster. Harvard researchers Fiona Chen and James Stratton took that intuition apart with engineering data aggregated by Jellyfish. After firms introduced AI coding agents, code output rose sharply, but software delivery, counted in Issues and Epics, did not rise with it. The extra efficiency was absorbed downstream, in code review.[1]
This post puts that study next to several things that happened around the same weeks of October 2026. QGIS changed its AI contribution policy to “not permitted,” while Linus Torvalds openly used Claude Code on a kernel patch. Rich Hickey argued that LLMs are automation, not tools, and Diomidis Spinellis predicted that thinking will become a hobby. DHH had an agent rewrite Campfire in Rust, and under heavy load it delivered about 1% of new-message notifications. One mechanism runs under all of these: the writing end gets wider, the review end gets longer, and who carries responsibility decides which governance fits.
Earlier posts on this site covered how to verify a coding agent’s “done” claims, which is about the evidence behind a single completion, and three habits that make coding agents burn money, which is about waste inside a trajectory. This one is about the organization: what to measure, who reviews, and when to open or close the gate.
Get the numbers straight: code went up, delivery didn’t
The study covers 2021 through March 2026: more than 700 software firms, over 700,000 employees, and about 300 million work events such as commits and pull requests, plus issue-tracker data from those firms. The researchers used directly measured AI usage and GitHub activity to date when each firm started using AI coding assistants or coding agents. Then they ran a difference-in-differences regression, comparing firms before and after adoption and early adopters against late ones.[1] The headline results:
| Metric | Average change after agents arrive |
|---|---|
| Total lines of code | +30% |
| Commits | +20% |
| Pull requests | +23% |
| Review time, PR submitted to merged | +49% |
| Share of PRs with changes requested | nearly doubles |
| Comments per PR | +35% |
| Share of employees doing code review | +14% |
| Issue and Epic resolution rate | no statistically significant change |
The writing end expanded, the review end dragged, and delivery barely moved. That is what a productivity paradox looks like in data. You might suspect that tickets got sliced thinner, so the resolution rate only looks flat. The researchers checked. They found no “compositional shift” in the size or complexity of tracked issues across adoption, so that explanation does not hold.[1]
Figure: writing metrics rise, review burden rises, and Issue and Epic resolution barely moves. Data from Chen and Stratton, as reported by Ars Technica.[1]
Screenshot: headline and dek of the Ars Technica report on the Chen and Stratton study (2026-10-09).[1]
By March 2026, about 95% of firms in the sample had deployed some kind of coding agent, and about 80% had used some form of AI code review. Yet agents accounted for only about 23.3% of review comments and about 10.8% of pull requests. Humans still carried most of the review work.[1] Put differently, handing review to the same class of model has not yet closed the gap that agent-written code opened. On employment, after cross-checking LinkedIn data the researchers could not attribute significant headcount changes to AI. Firms did not hire fewer people because they wrote more code, and they did not ship more software either.[1]
My view is blunt. If your team’s KPI is still “how many lines or PRs did we merge this week,” you are rewarding the widest part of the pipe, not whether the software actually moved forward. Measurement should follow the bottleneck.
What this data can and cannot tell you
Set the boundaries first, so an average doesn’t get read as a law.
First, this is an average effect across 700-plus firms. Difference-in-differences tells you how much more time adopting firms spent on review on average. It doesn’t tell you that your team will see the same thing. Some teams review fast and ship more. Some do worse. The value of the average is that it shows “write faster, ship faster” is not the default outcome. You have to prove it with your own numbers.
Second, the data stops in March 2026. Models and agents have improved noticeably since then, and many firms are still learning which tasks to hand to an agent.[1] Review burden might fall as teams gain experience. It might also keep rising as agents open more PRs in parallel. Both directions are plausible, so I read this as a snapshot of the current state, not an end state.
Third, the study covers both assistants that autocomplete human-written code and agents that write and submit code on their own, and the numbers in the table describe the change after agents arrive.[1] Autocomplete makes a person type faster, and the code still mostly passes through the author’s head. An agent hands over a whole PR, and the author understands each line far less well. The second puts much more pressure on review, which fits the mechanism.
The mechanism: the bottleneck moved from typing to trust
Coding agents don’t change whether code can be written. They change who produces something submittable, and how fast. A human author usually does a layer of self-review while writing, and gets stuck where their own thinking is unclear. An agent doesn’t get stuck. It can open several PRs in a row and push the self-review that should have happened during writing onto whoever reviews next. Review queues grow, the rework rate rises, and comment threads get longer. The +49%, the near-doubling, and the +35% in the study are the traces this mechanism leaves.[1]
Figure: the writing end widens and the review end lengthens. AI review has not yet replaced human review as the main path.[1]
This also explains the gap between “it worked locally” and “it’s on main.” Earlier posts here covered fake completions, receipts, and independent reruns: an agent saying tests passed does not mean the system is still correct after merge.[2] At the organization level it is the same problem, scaled up. Local speedups are not global throughput. When Epics in Jira don’t move, it isn’t because engineers got lazy. It’s because the trust needed to merge did not scale with the code.
What reviewers actually review
If most agent code runs, why does review get so much slower? Piotr Sarnacki’s teardown of DHH’s Campfire rewrite gives a concrete answer.[8] DHH didn’t want to read or write Rust himself, so he had an agent rewrite Campfire from Rails to Rust, then added Elixir and Go versions. Sarnacki’s observation is that when a prompt isn’t specific enough, many decisions become a coin flip, because they have no single correct answer. Do we keep backward compatibility? Do we favor latency or throughput? How much memory can the system use under load? Is losing new-content notifications after a crash acceptable? You may never have thought about these, but the agent answers every one of them, quietly, when it implements what you think you asked for.[8]
So the Rust version dropped the CSRF token to make caching easier and replaced Redis with in-process queues, while the Elixir version stayed much closer to the Rails original. The two rewrites made different compatibility trade-offs, which makes comparing the languages nearly meaningless.[8] Look further and some Rust database operations and locks block inside async tasks. Under cooperative scheduling, a task that doesn’t yield leaves every other task on that worker thread waiting. Under heavy load the Rust version delivered about 1% of new-message notifications, because the broadcast channel capacity was set to 256 and lagging receivers got cut off. Raising it to 16384 lifts delivery at 100 requests per second from about 14% to about 90%, but worst-case delivery latency goes from 11 seconds to over 130.[8] Which is better depends on whether you’d rather disconnect a slow client and let it reconnect, or make it wait three minutes for an update. That is a product decision, not a syntax question.
Figure: questions the prompt leaves open get answered by default in the implementation, and review has to find each of those defaults.[8]
Screenshot: the opening of Piotr Sarnacki’s “I’m sorry, but you still have to think,” on DHH’s agent-driven Campfire rewrite (2026-10-11).[8]
This example turns the +49% from the Harvard data into something you can touch. Facing an agent PR, a reviewer has to do more than check whether the code is correct. They have to dig out a string of defaults that nobody signed off on, then judge each against product intent. With human-written code, those decisions were often discussed before writing started, or the author can explain them on the spot. With agent-written code, the reviewer is often the first person to notice that a decision was made at all. Review work shifts from checking to filling in the design.
There is one more cost. Trajectory studies show that much of an agent’s spend goes to repeated searches, near-duplicate scripts, and idle test reruns that look reasonable in isolation.[3] Those actions produce more diffs and more surface work for humans to read. Rising code volume and slowing review are often two symptoms from the same pipe, one upstream and one downstream.
Maintainers split: QGIS closes the gate, Torvalds uses it himself
Around the same week, open source gave two nearly opposite governance answers.
QGIS changed its AI policy in QEP428 to a strict “not permitted”; PR #387 merged on 2026-10-07.[4] The maintainers wrote it plainly. AI use by outside contributors puts unsustainable pressure on review bandwidth that is already tight. AI output arrives faster than humans can review it carefully, so either unreviewed PRs pile up or review quality drops. A free software community is built on a network of people, and the organizations that trust and fund the project want contributors who can be hired for support and training. So the maintainers want to protect the bandwidth that welcomes human contributors.[4] The proposal’s author stressed that this is not a permanent ban. The door closes first and may reopen case by case. Enforcement relies on contributor honesty (“you know whether you wrote it yourself”), not on a repository scanner.[4]
Linus Torvalds, meanwhile, opened a branch called noref-filemap-read-with-llm in his own torvalds/linux.git. He used Claude Code to push a change that reads from the page cache without holding a folio reference, aimed at reducing folio refcount contention when many threads do small reads of the same file.[5] His patch notes say the code, the test program, the listed reviews, and the first draft of the cover letter were all done by Claude Code under his direction, with measurements run on his own machines.[5]
Figure: QGIS closes the gate to protect review bandwidth. Torvalds uses Claude Code under his own direction and acceptance.[4][5]
Screenshot: QGIS Enhancement Proposals #387 (QEP428), merged, setting the policy to no AI contributions.[4]
Screenshot: Phoronix reports the noref-filemap-read-with-llm branch in Torvalds’ tree.[5]
These two choices don’t cancel each other out. QGIS faces a bandwidth crisis of strangers contributing to a small set of maintainers. It is the same constraint as “review got longer” in the Harvard data, except an open source project has no budget to hire more reviewers. Torvalds faces a single high-leverage change he owns himself. He directs it, he accepts it, and he publicly scopes how much the LLM did. The bottleneck is still him. Only the speed of typing and drafting changed.
So “even Torvalds uses it” is not a license for a company to let agents open PRs everywhere. He didn’t outsource kernel review to a crowd of strangers’ agents. He gave himself a very fast typist, and review and sign-off still came from the person who knows that code better than anyone alive. If you move this case into a 200-person engineering org, the part to copy is the structure of who directs and who vouches, not the surface fact that Claude Code was involved.
The QGIS decision shouldn’t be read as anti-technology either. In a volunteer-maintained project, review time is the scarcest resource, and anything that raises review cost has to earn its place first. QGIS simply admitted that earlier than most companies have. They have no budget to pretend the pipe is infinitely wide.
Cognitive offloading: tools and automation are different things
Beyond measurement and governance, there is a harder question that doesn’t fit in a KPI: what skills do people keep inside this pipeline?
Rich Hickey put it flatly in a short essay on 2026-10-12: an LLM is not a tool, it is automation.[6] A tool is something you hold and drive with continuous control and sensory feedback, and using a tool builds expertise about the world and the task. Automation does the work itself, with little direct control from you. He compares the drive-thru, automation that spares you from cooking, with LLMs, automation that spares you from writing code by handing the writing away. You make a request and it acts on its own. That request-and-execute loop is neither driving nor continuous feedback.[6] When you use automation to read papers or write essays, you miss more than a few summaries. You miss learning how to argue, how to notice the edge cases, how to decide which authors to trust or doubt, and how to think.[6]
Diomidis Spinellis takes a longer view. He expects AI to take over most of today’s cognitive work, and human thinking to become like hunting, calligraphy, woodworking, or riding horses: an optional hobby rather than a livelihood.[7] He isn’t celebrating. His advice is to keep up with and use AI progress while building a life that still means something outside work. His own challenge, he says, is to move from thinking for a living to making thinking part of a life worth living.[7]
Both arguments sound philosophical. A study with a UC Berkeley co-author gives them a measurable version. The team recruited 1,222 participants online for randomized controlled trials. In one, a group solved 15 basic fraction problems with ChatGPT open alongside, free to ask for help or even the answer, while another group worked alone. The AI group started out more accurate. After 12 problems the researchers took the AI away, and that group almost immediately stopped answering correctly, skipping and giving up far more often. SAT reading comprehension tasks gave the same result.[9] The researchers conclude that relying on AI for roughly 10 minutes noticeably weakens people’s ability to persist at hard problems.[9]
Screenshot: UC Berkeley Research coverage of the randomized controlled study (2026-10-09).[9]
Fraction problems are not code review, and the experiment doesn’t transfer directly to senior engineers. But it points in the same direction as the Harvard data. Review is the stage of the pipeline that most needs someone to persist through a hard-to-read diff. If more people on a team get used to asking and receiving an answer, fewer will be willing and able to review an agent PR end to end. Those people set the ceiling on delivery.
Hickey’s distinction also helps you read the Torvalds case. What matters isn’t whether he used an LLM. It’s whether he still holds the intent, checks the result, and owns the merge. That looks like automation placed inside his control loop, not the steering wheel handed to a request queue. Two people can use the same Claude Code, and one learns while the other loses skill. The difference is who is watching at the other end of the loop.
A decision framework: measure delivery before widening the writing end
Here is the argument turned into an order of operations. I don’t recommend starting with a debate about whether to ban agents. It’s safer to change measurement first, then pick governance by who carries responsibility.
1. Change the primary metric
Move the main dashboard off lines, commits, and PR counts and onto quantities that track software moving forward: Issues and Epics closed above an agreed quality bar, production incident rate, review queue time, post-merge rollbacks, and the share of changes that pass alone but break after merge. The row that didn’t move in the Harvard study, Issue and Epic resolution, is the exit you should watch. The rows that went up belong on a secondary chart at most.[1]
2. Gate by responsibility structure, not one policy for everyone
| Situation | Closer to | Recommendation |
|---|---|---|
| High-leverage change directed and accepted by a core maintainer | Torvalds model | Let the agent draft. Publicly scope its involvement. Write acceptance criteria before writing code |
| Flood of outside contributions, fixed number of reviewers | QGIS model | Restrict or pause AI code contributions. Protect the human contribution path first |
| Internal team with headcount to add reviewers | Most firms in the study | Widen the writing end only if you widen review too (reviewer hours, gates, independent reruns), or delivery won’t move |
| Exploration, prototypes, one-off scripts | Low risk | Open by default. Before merging to main, have another person review or verify in another environment |
3. Expand review before you expand writing
The study shows that longer review and more rework are the main ways the efficiency gets absorbed, and AI review so far covers only a small share of comments and PRs.[1] In practice, independent reruns before merge, receipts (command, exit code, commit hash), and an acceptance checklist for “done” claims come closer to real review capacity than adding another same-family model as an “AI reviewer.”[2] If you can add neither reviewer hours nor gates, restrict the entry point the way QGIS did instead of pretending the pipe is infinitely wide.
4. Write the implicit decisions into the task
The Campfire example shows that a large share of review time goes to finding defaults nobody signed off on.[8] Rather than having reviewers dig them out afterward, put the key constraints in the task when you assign it: compatibility requirements, latency versus throughput priority, memory ceilings, and whether to drop or queue on failure. If you can’t write those down, the ticket isn’t thought through yet. Handing it to an agent only pushes the problem into review, where it costs more.
5. Protect the people who can still review
Hickey, Spinellis, and the Berkeley experiment all warn about skill decay. Keep a share of work that a human must read through and whose design trade-offs a human must explain. That isn’t ritual. It is the training load that keeps review ability alive. Hand all reading and writing to the request-and-execute loop and the dashboard looks great in the short run. Six months later you may not have qualified reviewers left.
6. Run an experiment window, not a slogan
Pick one product line and run a 4-to-6-week comparison. Take the same kind of Epic, keep one group on the old process, and let the other have agents open PRs. Use the same delivery metrics and review time for both. If code volume rises, delivery stays flat, and review gets longer, the conclusion is not “AI is useless.” It is “the bottleneck is review,” which matches the Chen and Stratton average.[1] With that reading you can decide whether to add reviewers, change gates, or narrow which tickets agents may take. And if delivery does go up, look at which kinds of tasks drove it, and spread that practice instead of the vague instruction to “use agents.”
Closing: software is capped by trust throughput
Coding agents are very good at producing more submittable text. Software moves forward through a much narrower channel: someone understands the change, is willing to vouch for it, and can show that it still holds in the merged system. The Harvard data says that, on average, this channel hasn’t widened yet. QGIS chose to protect the channel from being flooded. Torvalds chose to type faster while keeping his own hands on the wheel. Sarnacki’s teardown shows that what flows through the channel is not only code but a chain of decisions nobody made on purpose. Hickey, Spinellis, and the Berkeley experiment warn that if you outsource thinking for long enough, fewer people at the far end will be able to read the code.
So stop celebrating agent adoption with “how much more code did we write this week.” Ask a shorter question: how many more tickets that really needed closing did we close, and is the review queue getting shorter or longer? If you can answer that clearly, your governance will settle into one row of the table above. If you can’t, you are probably still optimizing the widest and least valuable part of the pipe.
References
- Kyle Orland, “AI coding agents generate more code, but not more software,” Ars Technica, 2026-10-09 (reporting on Fiona Chen and James Stratton’s study of Jellyfish data): https://arstechnica.com/ai/2026/10/ai-coding-agents-generate-more-code-but-not-more-software/ ↩
- On this site: When a Coding Agent Says Done ↩
- On this site: Three habits that make coding agents burn money ↩
- QGIS Enhancement Proposals #387, Add QEP428: No ai contributions permitted (merged): https://github.com/qgis/QGIS-Enhancement-Proposals/pull/387 ↩
- Michael Larabel, “Linus Torvalds Working On A Hefty Linux Kernel Improvement With Claude Code,” Phoronix: https://www.phoronix.com/news/Linus-Torvalds-Claude-Code ↩
- Rich Hickey, “‘AI’ is Not a Tool,” LinkedIn Pulse, 2026-10-12: https://www.linkedin.com/pulse/ai-tool-rich-hickey-kjl8e ↩
- Diomidis Spinellis, “Thinking will become a hobby,” 2026-10-08: https://www.spinellis.gr/blog/20261008/ ↩
- Piotr Sarnacki, “I’m sorry, but you still have to think,” 2026-10-11: https://itsallaboutthebit.com/i-am-sorry-but-you-still-have-to-think/ ↩
- Jason Pohl, “Using AI for Just 10 Minutes Erodes Your Ability to Persist at Hard Things,” UC Berkeley Research, 2026-10-09: https://vcresearch.berkeley.edu/news/using-ai-just-10-minutes-erodes-your-ability-persist-hard-things ↩