Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHacker News AI→ original

AI agents added 30% more code for engineers, but review of changes took 49% longer

Harvard PhD students studied 718 companies: after adopting AI agents, engineers added 30% more code, yet the number of tasks reaching production did not grow. Meanwhile, the time to review a pull request — a bundle of changes that colleagues must approve before merging — rose from 7 to 10.5 days, a 49% increase. Code is being written faster, but the number of reviewers stayed the same. So when buying agents, calculate review capacity first, not generation speed.

AI-processed from Hacker News AI; edited by Hamidun News
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHacker News AI ↗
Editor-in-Chief
Sample725,938 employees, Jellyfish data for 2021–2026
Key metric #2Revision requests in pull requests rose from 13% to nearly 25%
Key metric #3AI wrote only 23.3% of review comments, even though roughly 80% of firms have AI reviewers

Imagine a team that has been given AI agents: drafts of code appear many times faster, and the dashboards turn green. But users of the new features never see them. This is the picture described in a working paper by two Harvard PhD students, Fiona Chen and James Stratton, based on data from 718 companies.

After adopting agents, engineers added roughly 30% more code (an average of +1,494 lines per person per month on top of a baseline of 5,017). But the number of completed tasks did not grow in a statistically significant way, and the pull request path, that is, the bundle of changes colleagues review before merging into the main branch, lengthened by 49%: from 7.03 to about 10.

5 days. The bottleneck moved from writing code to reviewing it.

What exactly was measured

The authors used data from Jellyfish, a company that sells analytics on engineering activity, and matched it with employment profiles from LinkedIn provided by Revelio Labs. The sample included 725,938 employees at 718 firms from January 2021 to March 2026, and only Jellyfish clients who agreed to research use of their data were included. Inside were 68.3 million Jira tasks, 200.9 million commits and 49.1 million pull requests. Meeting calendar data was available for only 213 of the 718 firms.

The method was a month-by-month comparison: how metrics changed at companies that had already adopted agents versus those that had not yet. Adoption dates, according to the authors, depended mainly on when a firm learned about the tools and how long procurement took: approvals from legal and security teams, pilots. In one company the tool was approved in a day; in another, an interviewee said it took six to twelve months. The authors argue these delays are not related to the firm's own growth trajectory, and they control for company size, since large firms adopted earlier. Any residual bias is more likely to overstate the benefit of AI than understate it.

Code grew, finished work did not

Commits rose by 20% and pull requests by 23%. The authors caveat that this is the incremental effect of agents at companies that were already using AI coding assistants, not an effect from a standing start.

Finished work, however, did not move. A completed task is counted as a Jira ticket whose code was shipped to production. Their number changed by 0.12 per person per month against a baseline of 3.67, which is indistinguishable from zero, and the confidence interval rules out an increase of more than 12%. The authors tested the obvious objection: maybe engineers took on larger tasks. They labeled 0.5% of tasks by size using GPT-4o and extended the estimates to the rest via embeddings. Average task size did not change, and pull requests did not balloon.

The review bottleneck

The share of pull requests with a formal request for changes rose by about 12 percentage points from a baseline of 13%, nearly doubling. Comments per pull request rose by 35%. The share of employees who perform reviews at all rose by 14% from a baseline of 29%, while the share of people writing code did not change.

AI reviewers did not relieve the backlog. By March 2026, around 80% of firms had tried such tools, but AI wrote only 23.3% of comments, and only 10.8% of pull requests received at least one comment from AI. One junior engineer interviewed said that a single senior could approve his changes, and that person managed to do so once a week. After outages caused by AI-generated commits, a rule emerged that such code must be checked by a senior, and the queue grew even longer.

The authors explain the numbers through two mechanisms. The first: more code arrives at the same number of reviewers, which can be fixed by reallocating people to review. The second: the rise in revision requests hints at weaker drafts, which becomes a question of the models' quality itself. Which mechanism dominates determines where the budget should go.

Limitations and conclusion

This is a working paper submitted as a job-market paper, with no sign of completed peer review. The current version is dated August 4, 2026; the first was dated January 7. Employment did not fall, however: the estimates rule out a decline in total employment of more than 2.9%, though this is a short-term effect. An independent NBER study using data on more than 500,000 GitHub developers shows a similar gap: a 240% rise in commits from autonomous agents translates into a 30% increase in real releases. Both studies have their caveats: the authors of the NBER paper are affiliated with Microsoft, which owns GitHub. Both works describe software development, not, say, legal work.

The practical takeaway is simple. Measure not the volume of generated code, but the time to an accepted result, the share of returns for revision, and reviewers' hours. Plan review capacity first, then scale generation. And keep a log of what reviewers fix: it will show whether the backlog is caused by the volume or the quality of the drafts.

Story

Key Questions Answered

What exactly was measured?

The authors used data from Jellyfish, a company that sells analytics on engineering activity, and matched it with employment profiles from LinkedIn provided by Revelio Labs. The sample included 725,938 employees at 718 firms from January 2021 to March 2026, and only Jellyfish clients who agreed to research use of their data were included. Inside were 68.3 million Jira tasks, 200.9 million commits and 49.1 million pull requests. Meeting calendar data was available for only 213 of the 718 firms.

What are the key details on Code grew, finished work did not?

Commits rose by 20% and pull requests by 23%. The authors caveat that this is the incremental effect of agents at companies that were already using AI coding assistants, not an effect from a standing start. Finished work, however, did not move. A completed task is counted as a Jira ticket whose code was shipped to production. Their number changed by 0.12 per person per month against a baseline of 3.67, which is indistinguishable from zero, and the confidence interval rules out an increase of more than 12%.

What are the key details on The review bottleneck?

The share of pull requests with a formal request for changes rose by about 12 percentage points from a baseline of 13%, nearly doubling. Comments per pull request rose by 35%. The share of employees who perform reviews at all rose by 14% from a baseline of 29%, while the share of people writing code did not change. AI reviewers did not relieve the backlog.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…