LangSmith Shows What One Merged PR from Coding Agents Costs
Companies are spending more and more on coding agents, but most executives don't know what they are actually buying with those tokens. LangChain has added reporting for these agents to LangSmith: every cost in a trace is linked to a pull request on GitHub or GitLab. Teams can now see what one merged PR cost and how much money went to sessions that never led to a PR. Practical advice: start with a single repository and compare the cost per PR before and after a model change.
AI-processed from LangChain Blog; edited by Hamidun News
Spending on coding agents is growing faster at many teams than their understanding of what that money is buying. The provider's invoice shows tokens, but a manager needs an answer to a different question: how much does one unit of finished work cost? LangChain has tried to close that gap in LangSmith by adding reporting designed specifically for coding agents.
What was added
LangSmith now has a concept called outcomes. It links a pull request on GitHub or a merge request on GitLab to the sessions (threads) in which an agent worked on it. The cost of each trace is automatically distributed across the PRs the agent contributed to.
The link is not necessarily one-to-one. One session can produce several PRs, and one PR can take work across several sessions. The system accounts for both cases, so the numbers reflect actual work rather than a simplified model.
Coding agents have become the first specialized agent type in LangSmith. Each type has its own dashboard, its own connected data, and its own way of evaluation. Previously the platform showed the cost of an individual trace. That was enough for manual debugging but not for understanding trends across a whole team. The new overview page brings these figures together.
What the dashboard shows
The main metrics are placed on the first screen, and clicking any metric opens its trend over time. If a team has switched models, it can compare the periods before and after.
Spending is broken down by PR state: open, closed without merging, or merged. This shows how much money turned into accepted code and how much went into closed or stalled tasks.
A separate bucket is called unattributed, meaning spend not tied to any PR. It includes sessions in which the agent reviewed code or investigated a problem without creating a PR. This is not necessarily wasted money, and clicking through lets you open those sessions and check what actually happened.
Sessions are automatically labeled by task type: new features, bug fixes, investigations, reviews. For each type, the dashboard shows the share of sessions and the share of cost. What counts as irrelevant is up to the organization. For example, a team that uses the agent only for backend work might flag requests like "write ad copy."
The dashboard also shows which skills agents invoke, the most expensive PRs, and how cost grows with conversation length. Every metric leads to the specific conversation behind it.
How to use it
LangChain suggests several scenarios:
- Learning within the team. If one engineer gets merged PRs noticeably cheaper than another's, it's worth finding out what they do differently and sharing it. Typical causes of expensive PRs are a session that kept growing or a top-tier model run for a minor edit.
- Short sessions. Each new turn in a conversation carries the entire accumulated context, so the money concentrates in very long sessions. The habit of keeping one conversation open across several tasks is costly. The authors note that Anthropic also advises clearing context between tasks, and the dashboard will show whether habits change after such advice.
- Model selection. When a new model is released, you can compare the cost of a merged PR before and after the switch. A new provider can be run in parallel with the current one and compared on price and results in a single window. If subagents are moved to a cheaper model, you can see both the savings and whether PRs keep getting merged.
- Skills. If a particular skill keeps appearing in long, expensive sessions, its instructions may be sending the agent through unnecessary steps. If it appears most often in short sessions with a merged PR, it's worth rolling out to others.
- Talking to finance. The cost of a merged PR shows what a unit of completed work costs. Spend without a PR sits alongside it, so a high price doesn't always mean a bad process.
Agent support and launch
Many teams don't rely on a single tool: some work in Claude Code, others in Cursor or Copilot, and platform teams build their own agents on open frameworks like Deep Agents. LangSmith collects traces from all of them in one place, instead of scattering figures across different vendors' consoles.
The feature is available in public beta in LangSmith Cloud, and coding agent traces count against the regular trace limit. Setup takes three steps: install the LangSmith plugin for each agent you use, open the coding agents overview page, and wait for the first sessions. In Claude Code, the plugin installs with two /plugin commands inside a session, after adding the langchain-ai/langsmith-claude-code-plugins marketplace.
What to keep in mind
This material comes from LangChain itself, and it contains no independent measurements of savings. A merged PR remains a rough measure of value: it says nothing about code quality, and agents do review and research work that never produces a PR. The metric is useful as a baseline. The most practical approach is to start with one agent and one repository, look at the breakdown by PR, and only then compare models and policies.
- Nov 21, 2026Калифорния засудила OpenAI за ИИ-агентов, взломавших Hugging Face: штраф $2500 за нарушение
- Nov 21, 2026Anthropic Heads to IPO at $1.4 Trillion, but Seven Founders Keep 50.1% of the Votes
- Nov 21, 2026A Ghanaian medical AI with its safeguards removed designed 1,020 proteins and 5,000 toxin molecules
- Nov 21, 2026A game without AI outranked 2,782 projects in Epic Games' AI jam popularity ranking
Key Questions Answered
What was added?
LangSmith now has a concept called outcomes. It links a pull request on GitHub or a merge request on GitLab to the sessions (threads) in which an agent worked on it. The cost of each trace is automatically distributed across the PRs the agent contributed to. The link is not necessarily one-to-one. One session can produce several PRs, and one PR can take work across several sessions. The system accounts for both cases, so the numbers reflect actual work rather than a simplified model.
What the dashboard shows?
The main metrics are placed on the first screen, and clicking any metric opens its trend over time. If a team has switched models, it can compare the periods before and after. Spending is broken down by PR state: open, closed without merging, or merged. This shows how much money turned into accepted code and how much went into closed or stalled tasks. A separate bucket is called unattributed, meaning spend not tied to any PR.
How to use it?
LangChain suggests several scenarios: Learning within the team. If one engineer gets merged PRs noticeably cheaper than another's, it's worth finding out what they do differently and sharing it. Typical causes of expensive PRs are a session that kept growing or a top-tier model run for a minor edit. Short sessions. Each new turn in a conversation carries the entire accumulated context, so the money concentrates in very long sessions. The habit of keeping one conversation open across several tasks is costly.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.