AI agents in code: why a polished PR can hide production problems

Coding agents generate code at unprecedented speed. But a polished PR and a green CI run do not guarantee safety in production. Inside Vercel, the team developed a framework: it calls for disciplined work with AI instead of blind trust in tests. The structure includes gated pipelines, canary deployments and automatic rollback.

AI-processed from Vercel Blog; edited by Hamidun News
AI agents in code: why a polished PR can hide production problems
Source: Vercel Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceVercel Blog ↗
Editor-in-Chief
Self-driving deploymentsevery change rolls out incrementally through gated pipelines
Key metric #2Canary deployment with automatic rollback on degradation
Key metric #3Verification stages that cannot be bypassed

Coding agents generate code at unprecedented speed. In skilled hands, this is a powerful productivity multiplier, but without discipline it is simply an efficient way to ship wrong assumptions straight into production.

Why a green CI does not guarantee safety

AI code looks convincing. The PR description is tidy, static analysis raises no complaints, tests are green, and the code follows repository conventions. On the surface, it looks like the work of an experienced engineer.

But a green CI is no longer proof of safety. In the AI era, it merely reflects the agent's ability to convince your system that a change is safe, even if it instantly degrades infrastructure at scale.

The agent does not know your production. It does not know traffic patterns, failure modes or implicit infrastructure constraints. It does not know that Redis is nearing capacity, that the database is tied to a specific region, or that a flag rollout will shift the load on a downstream service.

The examples are not theoretical: a query that passes tests but will scan every row of the production database. Retry logic that seems correct but triggers a thundering herd on a dependent service. A cache without a TTL that quietly grows until Redis dies.

The gap between "looks right" and "safe to ship" has always existed. Agents widen it, because they produce code that looks more flawless than ever while remaining completely blind to production reality.

Use AI, don't rely on it

There is a fundamental difference between relying on AI and using it.

Relying means assuming: the agent wrote it, the tests passed, it's ready to ship. The author never builds a mental model of the change. The result is massive PRs with implicit assumptions that cannot be reviewed, because neither the author nor the reviewer sees the full picture.

Using means the agent helps you iterate quickly, but full responsibility for the output stays with you. You know how the code behaves under load. You understand the risks. You are ready to accept them. When you sign off on a PR, you are saying: "I have read this and I understand what it does."

If you need to reread your own PR to explain its impact on production, the engineering process has failed. The test is simple: are you ready to own a production incident tied to this PR?

How to protect production

The answer is not to stop using agents. The productivity gain is undeniable, and models keep getting better. AI assistants for code review and analysis are remarkably powerful tools that catch bugs and surface-level risks better than humans do.

But relying on review alone is a losing battle against the scale of agent-generated code. We have hit an inflection point where code implementation is abundant. The scarce resource is no longer writing code, but judgment about what is safe to ship.

The entire infrastructure must reflect this new reality. What is needed is a closed system in which agents act with high autonomy, because the environment is standardized, verification is trivial and deployment is safe by design.

  • Self-driving deployments: every change rolls out incrementally through gated pipelines
  • Canary deployment with automatic rollback on degradation
  • Verification stages that cannot be bypassed
  • Production insights built into the process, not bolted on at the end

The idea is simple: make the right decision the easiest one.

What this means

Development is moving from blind trust in tools to a controlled process. This is not about extra approvals; it is about making the AI agent work in conditions that guarantee safety.

For teams, this means preparing not for agent features, but for the discipline and structure around them.

Story

Key Questions Answered

Why a green CI does not guarantee safety?

AI code looks convincing. The PR description is tidy, static analysis raises no complaints, tests are green, and the code follows repository conventions. On the surface, it looks like the work of an experienced engineer. But a green CI is no longer proof of safety. In the AI era, it merely reflects the agent's ability to convince your system that a change is safe, even if it instantly degrades infrastructure at scale. The agent does not know your production. It does not know traffic patterns, failure modes or implicit infrastructure constraints. It does not know that Redis

What are the key details on Use AI, don't rely on it?

There is a fundamental difference between relying on AI and using it. Relying means assuming: the agent wrote it, the tests passed, it's ready to ship. The author never builds a mental model of the change. The result is massive PRs with implicit assumptions that cannot be reviewed, because neither the author nor the reviewer sees the full picture. Using means the agent helps you iterate quickly, but full responsibility for the output stays with you. You know how the code behaves under load. You understand the risks. You are ready to accept them. When you sign off on a PR, y

How to protect production?

The answer is not to stop using agents. The productivity gain is undeniable, and models keep getting better. AI assistants for code review and analysis are remarkably powerful tools that catch bugs and surface-level risks better than humans do. But relying on review alone is a losing battle against the scale of agent-generated code. We have hit an inflection point where code implementation is abundant. The scarce resource is no longer writing code, but judgment about what is safe to ship. The entire infrastructure must reflect this new reality. What is needed is a closed sy

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…