LangSmith Introduces LLM Gateway: Cost Control and PII Protection for AI Agents

LangSmith, a platform for developing and debugging LLM applications from LangChain, has introduced a built-in LLM Gateway. This new tool allows developers to control spending on large language model calls, protect user data, and track each call in real time. Management happens directly from the development environment, without installing additional services.

How Gateway Works

LLM Gateway is built directly into LangSmith and operates at runtime—at the moment a request is executed. The tool sits between the agent code and the LLM provider service (OpenAI, Anthropic, Google, and others), intercepts each request, checks it against established rules, and can apply necessary transformations before sending it to the server. Previously, a developer had to either write custom middleware or rely on logging after request execution—after the fact, so to speak.

Now Gateway integrates automatically, without changes to the application code. This simplifies production deployment and makes control more reliable. For large organizations with multiple AI applications, this built-in nature is critical.

The platform becomes not just a logging system, but an active tool for managing LLM spending and security in real time.

Three Key Capabilities

Gateway focuses on three problems that developers face when working with AI agents:

• Cost and budget control—setting strict limits on token count or direct request cost, with varying granularity: at the level of an individual agent, user, entire organization, or API key
• Personal data protection (PII redaction)—automatic removal of sensitive information (credit card numbers, SSN, emails, physical addresses, document numbers) from prompts BEFORE sending to the server
• Complete tracing—saving detailed logging of each LLM call with information about model version, parameters, response time, and cost

The problem of personal data leakage is especially acute in production applications. In practice, a developer often cannot fully control which user data makes it into prompts—through APIs, logging, user input. Gateway acts as a filter: it clears dangerous values before sending and records the fact of redaction so that compliance and security teams know where the risk was and how it was handled. Financial control is also critical. Large language models are expensive, and complex agents often make a dozen calls in a single user session. When limits are set at the platform level (not manually in code), uncontrolled budget spending becomes impossible.

Built Into the Workflow

The main feature of Gateway is that it's not a separate microservice that needs to be installed and configured separately. It's a built-in component of LangSmith. A developer opens the platform dashboard, enables the necessary rules (spend limits, PII patterns, rate limits), and they start working immediately for all agents. This reduces the complexity of development and deployment. When governance is built into the main platform, the team doesn't spend time and resources on integrating and monitoring a separate system, and doesn't have to hunt for bugs at service boundaries.

This is the first governance tool that is built into the agent

lifecycle, not bolted on top.

What This Means

For developers: cost control and data protection are now automatic. Instead of writing checks into every request, it's enough to set rules once in LangSmith. The team can focus on agent logic while routine checks run themselves. For companies: it's safer and more cost-effective to scale AI applications in production. Financial risks are contained at the platform level, PII protection works out of the box, logging automatically assists with audits and compliance checks. LangSmith becomes a full-fledged LLM agent management platform—from a sketch in a developer's notebook to scaling in production with built-in management and governance.