Harvard's Coding-Agent Paper Moved From 49% to 56.1% Slower Review Between Drafts, and Issues Moved Too
Software / analysis
Harvard's Coding-Agent Paper Moved From 49% to 56.1% Slower Review Between Drafts, and Issues Moved Too
The Nov. 7 revision of the Chen and Stratton paper adds five months of data, and resolved issues shifted from no effect to a borderline 13.7% gain.

Harvard's working paper on AI coding agents now exists in two versions, and the headline number moved between them: pull-request time-to-merge rose 49% in the draft built on data through March 2026, and 56.1% in the Nov. 7 revision with data through August. The review bottleneck survived the update. The claim that agents add code but not finished work did not survive intact.
The paper is "Artificial Intelligence in the Firm: Bottlenecks in Software Production" by Fiona Chen and James Stratton, both Ph.D. candidates at Harvard, according to ComplexDiscovery's write-up of the earlier draft. It was the subject of a Hacker News thread that reached 84 points. Neither version has been through peer review.
What the earlier draft measured
The earlier draft covered 718 firms and 725,938 workers from January 2021 to March 2026, using data from the engineering-analytics company Jellyfish joined to LinkedIn employment records from Revelio Labs. It compared firms that had adopted coding agents with firms that had not yet, a staggered difference-in-differences design.
Output rose by roughly a quarter or more on every volume measure: lines of code added went up about 30% per worker-month, commits 20% and pull requests 23%. Resolved issues did not. The estimate was 0.12 issues per worker-month on a baseline of 3.67, with a standard error of 0.17, and the confidence interval ruled out a gain above about 12%.
Review absorbed the difference. Time from pull-request submission to merge grew by 3.45 days on a 7.03-day baseline. Review comments per pull request rose 35%, and formal change requests rose about 12 percentage points from a 13% baseline.
What the November revision changed
Beri's account of the Nov. 7 revision describes a sample of 720,542 engineers at 655 firms, with data through August 2026. That is fewer firms but more calendar. Every volume figure grew, and the one that tells the shipped-work story moved from "no effect" to "borderline".
| Measure | Earlier draft | Nov. 7 revision |
|---|---|---|
| Pull requests | +23% | +35.7% |
| Lines of code | +30% | +44.0% |
| Commits | +20% | +36.0% |
| Resolved issues | no significant change | +13.7%, borderline |
| Review comments per change | +35% | +38.6% |
| Submission-to-approval time | +49% (to merge) | +56.1% (to approval) |
The last row compares two different clocks: the earlier draft measured time to merge, and Beri reports time to approval. Treat the 49% and 56.1% as neighbours, not as one series.
- Earlier draft, data to March 202623 %
- Nov. 7 revision, data to August 202635.7 %
Source: ComplexDiscovery (earlier draft) and Beri (Nov. 7 revision), accessed 2026-11-18
Where the delay sits
Beri reports that the Nov. 7 revision splits the added review time in two: backlog wait for a reviewer rose 59.9% and active review time rose 58.5%. So reviewers are both slower to start and slower to finish. The revision also reports that 82.8% of reviews go to engineers already familiar with the code, and that senior-authored changes are reviewed by seniors 75.7% of the time against 32.8% for junior-authored ones.
That last pair is the least reported and most useful for a team lead. If agent output lands on the same few senior reviewers, adding agents adds work to the scarcest people. Tooling is arriving to feed that queue faster, such as the Superpowers skills framework that now installs into 16 coding agents, and JetBrains' ThinkRail, which wraps one agent, pi, in a worktree IDE.
AI review tools have not absorbed it yet. By March 2026, nearly 80% of firms had used one, yet AI wrote 23.3% of review comments and touched only 10.8% of pull requests, per ComplexDiscovery. Beri's 79.1% adoption figure for the revision matches, and it says humans still write most review comments.
The caveats that decide how far to trust it
Jellyfish sells the analytics the study draws on, and the earlier-draft write-up notes that the authors have published on its blog. Beri adds that Jellyfish reviewed the manuscript only to check for proprietary disclosures. The firms in the sample use an analytics vendor, which may make them more process-heavy than a typical team, so review could bind harder there than elsewhere.
Jira issue counts can miss real value such as tests and refactors. Individual-licence AI use goes unrecorded, per the earlier draft's write-up. And in the earlier draft only 213 of 718 firms supplied calendar data.
The employment finding also moved. The earlier draft ruled out an overall employment decline larger than 2.9%. The revision reports a 2.6% fall in active engineers with an interval that rules out declines larger than 15.6%, which is a looser bound. Beri also flags that the revision's own projections for employment disagree between its body and its FAQ, giving 13.2% in one and 10.3% by 2031 in the other under different assumptions.
What would change the reading
The review story holds if the next revision keeps merge and approval times up by half while resolved issues stay near the +13.7% line. It weakens if resolved issues climb toward the 44% lines-of-code figure, which would mean review is slow but not binding. Teams that want to test it locally have a concrete metric: median hours from pull request opened to approved, split into wait and active time, before and after an agent rollout.
One caution on sourcing: this piece draws the revision's figures from Beri's account, not from the revised PDF, which could not be read in text form. The first thing to check in the paper is the sample note, 655 firms against 718, because the two samples may not overlap.
Sources
More in Software
- 01OpenAI's WebRTC Voice Stack Draws a 30 ms Objection That Its Own Engineers Never AddressedLuke Curley says WebRTC drops audio older than 30 ms. OpenAI's engineers wrote about port ranges and routing, which is a different problem.
- 02Superpowers Passes 322,000 GitHub Stars as a Set of 15 Prompted Skills for 16 Coding AgentsJesse Vincent's MIT-licensed framework tells agents to brainstorm, plan and test first. Its clone count is large, and its evidence for better results is one author's view.
- 03GitHub Disables 144 Picofly Repositories After a Nintendo Notice Names OneThe November 6 notice is the first on GitHub aimed at Switch modchip firmware, and it asked for forks to be taken down without listing a single one.
- 04Chrome's Gemini Nano Download: What Google's Docs Say, and the Numbers They Leave OutGoogle's page on built-in model management gives triggers, a 30-day rule and full-download updates, but no file size, no disk threshold and no consumer off switch.