Cohere explains why AI ROI is hard to calculate, and how to do it

Cohere has published a post on its corporate blog explaining how companies should calculate the return on AI investment. The core of the calculation fits on one line: ROI = (financial benefit − total cost) ÷ total cost × 100. The whole difficulty lies in what goes into the numerator and the denominator. The piece was written by product marketer Ariana Milligan, so it is a vendor's perspective. But most of its arguments are not tied to any specific model provider.

What counts as benefit and what counts as cost

Cohere defines AI ROI as the financial return on an investment relative to the total cost of deploying and operating it. The return comes in two ways: through savings or through additional sales revenue. The key caveat is that operational improvements are not money in themselves. A faster task, higher output or better-quality work can produce a financial effect, but ROI turns positive only if that effect exceeds the cost of achieving it.

Two things complicate the picture. Costs are spread across technical infrastructure and operational resources, and the benefit is hard to isolate, because AI is usually embedded in broader business processes. It is not always possible to pin down its contribution to the final result.

Four sources of gaps between forecast and reality

The author lists the features of AI investments that most often break the calculation. In the published portion of the text, there are four:

• Unit economics depend on workload and scale. Longer context, more reasoning, and more calls to the model and to tools make a task more expensive. Agentic systems vary even more: similar tasks take a different number of steps and retries. Per-task costs in a pilot may not match production, where the load is larger and more varied. Growing usage raises variable costs, but it can also improve utilization of capacity that has already been paid for.
• Choice of model and deployment setup shifts the balance between price and quality. Expensive models are needed for complex tasks, while cheap ones are often enough for simple ones. Cohere highlights an approach it calls cost-aware routing: the application uses several models and sends each task to the cheapest one that meets the quality requirements.
• Deployment affects the cost structure. Managed pay-as-you-go services are more economical when demand is small or unpredictable. Dedicated capacity, whether hosted by a provider or owned, delivers better unit economics under high, steady load, but sits idle when demand is weak or uneven. Beyond price, latency, data residency, security and operational control also factor into the decision.
• Quality and reliability requirements increase operating costs. Monitoring output stability, running automated checks and handing doubtful or risky cases to a human add costs for compute, tooling and labor. For agents, the evaluation has to cover the chain of tool calls, not just the final answer.

There is one more trap: a gain at the level of a single task can overstate the financial effect for the whole organization. If another stage of the process remains the bottleneck, the speedup simply moves the jam elsewhere, and end-to-end throughput does not grow. The available text cuts off before this section ends.

How to apply this in practice

What follows is our editorial takeaway from the logic of the piece, not quotes from Cohere.

First: count the cost of a completed business task, not the price of a token. For agents, use the distribution rather than the average: the median and the tail of steps and retries show the real budget.

Second: recalculate the pilot at production load before approving a budget. Include evaluation, monitoring and manual review in the denominator, otherwise ROI will come out overstated.

Third: before booking time savings as benefit, check where the next bottleneck in the process is. If the throughput of the whole process has not grown, you have not saved any money yet.

Fourth: compare deployment options against your demand profile. Pay-as-you-go suits unstable load, while dedicated capacity is worth costing out for dense, constant load. Constraints on latency, data and security may rule out some options before you ever compare prices.