AI agents in code: why a polished PR can hide production problems
Coding agents generate code at unprecedented speed. But a polished PR and a green CI run do not guarantee safety in production. Inside Vercel, the team developed a framework: it calls for disciplined work with AI instead of blind trust in tests. The structure includes gated pipelines, canary deployments and automatic rollback.
AI-processed from Vercel Blog; edited by Hamidun News
Coding agents generate code at unprecedented speed. In skilled hands, this is a powerful productivity multiplier, but without discipline it is simply an efficient way to ship wrong assumptions straight into production.
Why a green CI does not guarantee safety
AI code looks convincing. The PR description is tidy, static analysis raises no complaints, tests are green, and the code follows repository conventions. On the surface, it looks like the work of an experienced engineer.
But a green CI is no longer proof of safety. In the AI era, it merely reflects the agent's ability to convince your system that a change is safe, even if it instantly degrades infrastructure at scale.
The agent does not know your production. It does not know traffic patterns, failure modes or implicit infrastructure constraints. It does not know that Redis is nearing capacity, that the database is tied to a specific region, or that a flag rollout will shift the load on a downstream service.
The examples are not theoretical: a query that passes tests but will scan every row of the production database. Retry logic that seems correct but triggers a thundering herd on a dependent service. A cache without a TTL that quietly grows until Redis dies.
The gap between "looks right" and "safe to ship" has always existed. Agents widen it, because they produce code that looks more flawless than ever while remaining completely blind to production reality.
Use AI, don't rely on it
There is a fundamental difference between relying on AI and using it.
Relying means assuming: the agent wrote it, the tests passed, it's ready to ship. The author never builds a mental model of the change. The result is massive PRs with implicit assumptions that cannot be reviewed, because neither the author nor the reviewer sees the full picture.
Using means the agent helps you iterate quickly, but full responsibility for the output stays with you. You know how the code behaves under load. You understand the risks. You are ready to accept them. When you sign off on a PR, you are saying: "I have read this and I understand what it does."
If you need to reread your own PR to explain its impact on production, the engineering process has failed. The test is simple: are you ready to own a production incident tied to this PR?
How to protect production
The answer is not to stop using agents. The productivity gain is undeniable, and models keep getting better. AI assistants for code review and analysis are remarkably powerful tools that catch bugs and surface-level risks better than humans do.
But relying on review alone is a losing battle against the scale of agent-generated code. We have hit an inflection point where code implementation is abundant. The scarce resource is no longer writing code, but judgment about what is safe to ship.
The entire infrastructure must reflect this new reality. What is needed is a closed system in which agents act with high autonomy, because the environment is standardized, verification is trivial and deployment is safe by design.
- Self-driving deployments: every change rolls out incrementally through gated pipelines
- Canary deployment with automatic rollback on degradation
- Verification stages that cannot be bypassed
- Production insights built into the process, not bolted on at the end
The idea is simple: make the right decision the easiest one.
What this means
Development is moving from blind trust in tools to a controlled process. This is not about extra approvals; it is about making the AI agent work in conditions that guarantee safety.
For teams, this means preparing not for agent features, but for the discipline and structure around them.
- Nov 1, 2026Local models lose to Opus and Codex: 0.21 on a MacBook versus 1.00 for Codex
- Nov 1, 2026Adobe's EditAgent judge picks a usable edit 81.3% of the time, versus 74.8% for EditScore
- Nov 1, 202616 models tested on 7 search tools: Exa averaged 10.5 out of 14, SearXNG about 3
- Nov 1, 2026Google's StreamVibe video agent beats other models by 10.7% on response timing and quality
Key Questions Answered
Why a green CI does not guarantee safety?
AI code looks convincing. The PR description is tidy, static analysis raises no complaints, tests are green, and the code follows repository conventions. On the surface, it looks like the work of an experienced engineer. But a green CI is no longer proof of safety. In the AI era, it merely reflects the agent's ability to convince your system that a change is safe, even if it instantly degrades infrastructure at scale. The agent does not know your production. It does not know traffic patterns, failure modes or implicit infrastructure constraints. It does not know that Redis
What are the key details on Use AI, don't rely on it?
There is a fundamental difference between relying on AI and using it. Relying means assuming: the agent wrote it, the tests passed, it's ready to ship. The author never builds a mental model of the change. The result is massive PRs with implicit assumptions that cannot be reviewed, because neither the author nor the reviewer sees the full picture. Using means the agent helps you iterate quickly, but full responsibility for the output stays with you. You know how the code behaves under load. You understand the risks. You are ready to accept them. When you sign off on a PR, y
How to protect production?
The answer is not to stop using agents. The productivity gain is undeniable, and models keep getting better. AI assistants for code review and analysis are remarkably powerful tools that catch bugs and surface-level risks better than humans do. But relying on review alone is a losing battle against the scale of agent-generated code. We have hit an inflection point where code implementation is abundant. The scarce resource is no longer writing code, but judgment about what is safe to ship. The entire infrastructure must reflect this new reality. What is needed is a closed sy
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.