Anthropic releases FlashProve: proving model inference costs 70–95 times as much as inference itself

You buy access to an expensive model through an intermediary. The response arrives, the model is named, and the bill is issued. How can you check that this model actually processed your request? Text quality alone makes that difficult to establish reliably. On November 2, Anthropic introduced FlashProve, an open prototype that lets users verify the connection between a result and weights committed to in advance. For developers, its appeal is a way to audit a provider, although the cost currently prevents its use on every request. Anthropic announcement.

What the proof actually establishes

Imagine a service that promises to process documents with a particular model. The provider first publishes a cryptographic commitment: it binds the provider to a specific set of weights while concealing their values. It then supplies a proof alongside the computation result. The client verifies that proof on a CPU, without downloading the weights or running the large model again.

FlashProve is built on zkSNARK, a cryptographic protocol for verifying computations. In this scenario, the verifier receives confirmation that the claimed result corresponds to the committed model and the input data. The proof itself is designed not to reveal the weights. This separates two tasks: the provider stores the model and performs the heavy computation, while the client verifies the resulting evidence. The principle is described in the technical report.

One question remains, however: who confirmed that the published identifier actually belongs to the model you paid for? If an intermediary chooses cheap weights and labels them as an expensive model, a valid proof will not detect that deception. The identifier must be obtained in advance from a trusted party. This trust boundary is explicitly stated in the project README.

For a product team, this adds a requirement to its arrangements with the provider: receiving a proof file is not enough. The team needs to understand which reference it is checked against and who is responsible for that reference.

Why 110 tokens per second is easy to misinterpret

The experiment used Llama 4 Maverick with 4-bit weights and a context of 1024 tokens. With a batch of 24 requests, a single H100 processed around 110 input tokens per second while generating proofs. This measures prompt processing, not the delivery of newly generated tokens to the user. Each request in this benchmark proved one next token. Measurement conditions.

The distinction matters. The figure of 110 is easy to mistake for the speed of a full chat system with verifiable responses. In practice, a request in the batch waited for proofs for the entire batch to finish: approximately 221–226 seconds, before any additional verification. Throughput therefore cannot be translated into promises about user-facing latency. These are separate characteristics of the same experiment. Performance table.

Anthropic estimates that generating proofs requires roughly 70–95 times as much GPU time as ordinary inference through vLLM. The proof for the test request is around 19 MB. For an API operating at scale, this significantly changes computation and data transfer costs. Authors' estimate.

What to check before experimenting

The guarantees apply to a specific integer implementation of the model. Whether its responses match those of the standard floating-point version requires a separate assessment. Strict verification of an exact match and a mode that permits deviations also provide different guarantees. A program completing successfully does not, by itself, explain exactly what it has established. Verification also covers only the specified token positions. Known limitations.

My practical takeaway is to start by verifying a saved example and examining the verifier's output. The repository provides a way to do this without a GPU, and the code is released under Apache 2.0. The authors nevertheless describe the project as an unmaintained research artifact. FlashProve repository.

For a team building an AI API auditing system, the next useful step is to define its own requirements: which computations must be proved, who certifies the weights, and how long it can afford to wait. Then measure the cost in its own use case. Proof of a response's provenance does not, by itself, make that response semantically correct. It does, however, provide a technical way to verify a provider's promise that today often has to be taken on trust.