CodeRabbit Ships 20x More Code with Codex Without Hiring More Engineers

You rolled out coding agents, the number of pull requests jumped several times over, and suddenly your team is drowning in review. According to CodeRabbit CEO Harjot Gill, this is the first bottleneck any company hits after adopting agents. His startup builds its business on exactly this problem, and OpenAI recently published a breakdown of its case. This is vendor material, so the numbers should be read with a grain of salt. Still, it contains more engineering insight than a typical advertisement.

The bottleneck has moved to review

CodeRabbit builds a platform that checks code written by agents and helps decide what is safe to ship. It is used by more than 20,000 organizations and over 150,000 open-source projects. The first version of the product was built on GPT-3.5 Turbo, it became genuinely useful with GPT-4, and the team then moved early to o1 and o3.

The key point: review runs asynchronously in the cloud. An instant response isn't needed there, so you can use slow reasoning models and pay in time for depth of analysis. If your review also runs in the background, this trade-off is worth considering when choosing a model.

The team tests new models against its own set of known bugs. By Gill's estimate, GPT-6 Astra catches about 50% of the issues in it and leads among the models tested. The set is constantly updated: agents make new kinds of mistakes, and fixes reveal what was previously missed. A static benchmark quickly becomes outdated here.

Cheap subagents and an expensive orchestrator

At first, CodeRabbit had one or two subagents for navigating code. Now cheap models such as GPT-6 Luna serve many subagents across the platform, while GPT-6 Astra handles planning and orchestration. Both tokens and caching have become cheaper. Gill says agents can now be deployed where it used to make no economic sense.

This is the standard "smart planner plus many cheap executors" pattern. But here it is backed by a concrete conclusion: lower prices translate directly into deeper review, not just savings.

On security: CodeRabbit has been running agents in sandboxes since 2023, and through the Daybreak program the team gained access to OpenAI's Cyber model. They used it for internal pentesting and found vulnerabilities they "couldn't have imagined" themselves. This grew into the product CodeRabbit Secure. During a two-week promotion with a discount on $1 million worth of deep scans, the company found more than 43,000 vulnerabilities across 13,000 scans. Customers have already fixed thousands of them. That said, this is a marketing promotion, and it is unknown how many of the findings are actually critical.

What to take away, and with what caveat

The headline number: roughly 20 times more software shipped in six months without hiring. Gill personally estimated that he submitted 2,500–3,000 pull requests in 30 days. That is his own estimate, and "volume of software" is not the same as value to users.

More interesting is what had to change. Several parallel agents ran into the limits of laptop memory, tests, builds, and dependencies. The team simplified the release process, moved part of development to the cloud for access control and auditing, and decided where a human is needed. CodeRabbit merges low-risk changes automatically, checking them with its own product. Gill calls this "a complete rethink of software delivery."

There is also a Codex-based Slack bot that analyzes incidents, messages, and support tickets and returns a ready pull request. The company's marketing team runs Google Ads through pull requests and keeps slides and documents in a repository. The slogan is "run the company as code."

Practical takeaways for teams:

• Prepare review and CI for a growing number of PRs in advance, not afterward.
• Build your own bug benchmark and keep updating it.
• Separate risks: what is merged automatically and what goes to a human.
• Keep the expensive model for planning and the cheap one for bulk work.

The limitation is obvious: the company sells the very tool it praises, and there are no independent measurements. But the underlying logic, that the bottleneck moves from writing code to verifying it, applies just as well to your team.