Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseCode / SDK · ActionableLangChain Blog→ original

LangSmith Shows What One Merged PR from Coding Agents Costs

Companies are spending more and more on coding agents, but most executives don't know what they are actually buying with those tokens. LangChain has added reporting for these agents to LangSmith: every cost in a trace is linked to a pull request on GitHub or GitLab. Teams can now see what one merged PR cost and how much money went to sessions that never led to a PR. Practical advice: start with a single repository and compare the cost per PR before and after a model change.

AI-processed from LangChain Blog; edited by Hamidun News
LangSmith Shows What One Merged PR from Coding Agents Costs
Source: LangChain Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceLangChain Blog ↗
Editor-in-Chief
Key metric #1Claude Code, Cursor, Copilot, and the open-source Deep Agents are supported
Key metric #2Spend without a PR, such as reviews and code analysis, is tracked separately
Beta in LangSmith Cloudthe Claude Code plugin installs with three commands

Spending on coding agents is growing faster at many teams than their understanding of what that money is buying. The provider's invoice shows tokens, but a manager needs an answer to a different question: how much does one unit of finished work cost? LangChain has tried to close that gap in LangSmith by adding reporting designed specifically for coding agents.

What was added

LangSmith now has a concept called outcomes. It links a pull request on GitHub or a merge request on GitLab to the sessions (threads) in which an agent worked on it. The cost of each trace is automatically distributed across the PRs the agent contributed to.

The link is not necessarily one-to-one. One session can produce several PRs, and one PR can take work across several sessions. The system accounts for both cases, so the numbers reflect actual work rather than a simplified model.

Coding agents have become the first specialized agent type in LangSmith. Each type has its own dashboard, its own connected data, and its own way of evaluation. Previously the platform showed the cost of an individual trace. That was enough for manual debugging but not for understanding trends across a whole team. The new overview page brings these figures together.

What the dashboard shows

The main metrics are placed on the first screen, and clicking any metric opens its trend over time. If a team has switched models, it can compare the periods before and after.

Spending is broken down by PR state: open, closed without merging, or merged. This shows how much money turned into accepted code and how much went into closed or stalled tasks.

A separate bucket is called unattributed, meaning spend not tied to any PR. It includes sessions in which the agent reviewed code or investigated a problem without creating a PR. This is not necessarily wasted money, and clicking through lets you open those sessions and check what actually happened.

Sessions are automatically labeled by task type: new features, bug fixes, investigations, reviews. For each type, the dashboard shows the share of sessions and the share of cost. What counts as irrelevant is up to the organization. For example, a team that uses the agent only for backend work might flag requests like "write ad copy."

The dashboard also shows which skills agents invoke, the most expensive PRs, and how cost grows with conversation length. Every metric leads to the specific conversation behind it.

How to use it

LangChain suggests several scenarios:

  • Learning within the team. If one engineer gets merged PRs noticeably cheaper than another's, it's worth finding out what they do differently and sharing it. Typical causes of expensive PRs are a session that kept growing or a top-tier model run for a minor edit.
  • Short sessions. Each new turn in a conversation carries the entire accumulated context, so the money concentrates in very long sessions. The habit of keeping one conversation open across several tasks is costly. The authors note that Anthropic also advises clearing context between tasks, and the dashboard will show whether habits change after such advice.
  • Model selection. When a new model is released, you can compare the cost of a merged PR before and after the switch. A new provider can be run in parallel with the current one and compared on price and results in a single window. If subagents are moved to a cheaper model, you can see both the savings and whether PRs keep getting merged.
  • Skills. If a particular skill keeps appearing in long, expensive sessions, its instructions may be sending the agent through unnecessary steps. If it appears most often in short sessions with a merged PR, it's worth rolling out to others.
  • Talking to finance. The cost of a merged PR shows what a unit of completed work costs. Spend without a PR sits alongside it, so a high price doesn't always mean a bad process.

Agent support and launch

Many teams don't rely on a single tool: some work in Claude Code, others in Cursor or Copilot, and platform teams build their own agents on open frameworks like Deep Agents. LangSmith collects traces from all of them in one place, instead of scattering figures across different vendors' consoles.

The feature is available in public beta in LangSmith Cloud, and coding agent traces count against the regular trace limit. Setup takes three steps: install the LangSmith plugin for each agent you use, open the coding agents overview page, and wait for the first sessions. In Claude Code, the plugin installs with two /plugin commands inside a session, after adding the langchain-ai/langsmith-claude-code-plugins marketplace.

What to keep in mind

This material comes from LangChain itself, and it contains no independent measurements of savings. A merged PR remains a rough measure of value: it says nothing about code quality, and agents do review and research work that never produces a PR. The metric is useful as a baseline. The most practical approach is to start with one agent and one repository, look at the breakdown by PR, and only then compare models and policies.

Story

Key Questions Answered

What was added?

LangSmith now has a concept called outcomes. It links a pull request on GitHub or a merge request on GitLab to the sessions (threads) in which an agent worked on it. The cost of each trace is automatically distributed across the PRs the agent contributed to. The link is not necessarily one-to-one. One session can produce several PRs, and one PR can take work across several sessions. The system accounts for both cases, so the numbers reflect actual work rather than a simplified model.

What the dashboard shows?

The main metrics are placed on the first screen, and clicking any metric opens its trend over time. If a team has switched models, it can compare the periods before and after. Spending is broken down by PR state: open, closed without merging, or merged. This shows how much money turned into accepted code and how much went into closed or stalled tasks. A separate bucket is called unattributed, meaning spend not tied to any PR.

How to use it?

LangChain suggests several scenarios: Learning within the team. If one engineer gets merged PRs noticeably cheaper than another's, it's worth finding out what they do differently and sharing it. Typical causes of expensive PRs are a session that kept growing or a top-tier model run for a minor edit. Short sessions. Each new turn in a conversation carries the entire accumulated context, so the money concentrates in very long sessions. The habit of keeping one conversation open across several tasks is costly.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…