FineServe: датасет реальных нагрузок обслуживания LLM с глобального маркетплейса

Researchers have introduced FineServe — an open dataset of real-world LLM serving workloads, collected from a global commercial marketplace and published on arXiv in July 2026 (arXiv:2607.19349). Along with the dataset, the authors released a workload generator for benchmarking multi-model serving platforms.

What FineServe is

FineServe is an "in-the-wild" dataset of real server workloads captured from a live commercial LLM marketplace, where different models and tasks are served in parallel. Unlike synthetic data, it captures how requests actually arrive and how tokens behave across heterogeneous models.

• Type: multi-model dataset of LLM serving workloads
• Data source: global commercial LLM marketplace
• Publication: arXiv:2607.19349, July 2026
• Components: the dataset itself + a workload generator
• Access: open code and data on GitHub

Why proxy traces don't cut it

Existing LLM serving research relies on proxy traces or coarse-grained generalizations that fail to capture the heterogeneity of modern multi-model platforms. FineServe closes this gap: the authors analyzed request arrival dynamics and token behavior across three axes at once — model architecture, model scale, and task type — and found fundamentally different load fluctuation regimes.

This matters for engineers running LLMs as an always-on online service: low latency and high throughput under bursty demand require an accurate understanding of the actual traffic profile, not an averaged picture.

What the workload generator provides

FineServe's workload generator assembles fine-grained, "model-aware" profiles into configurable mixes tailored to a specific benchmark scenario. This means a team can reproduce workload close to production and use it to run routing, scheduling, and capacity-planning strategies.

"By revealing these fine-grained workload dynamics,

FineServe provides a realistic foundation for evaluating routing, scheduling, and capacity planning strategies in LLM serving systems," the paper's abstract on arXiv states.

In the authors' framing, the key value is precisely heterogeneity: different models and request types behave differently, and an averaged benchmark hides this.

What this means

For those operating multi-model platforms (and AlpinaGPT and similar aggregators are exactly this class of system), FineServe is a reason to double-check what traces your infrastructure is being tested on. A realistic workload profile directly affects routing choices and how many GPUs are actually needed to handle peak demand.

Frequently asked questions

What is FineServe?

FineServe is an open dataset of real-world LLM serving workloads plus a workload generator, presented on arXiv (arXiv:2607.19349) in July 2026. The data was collected from an operating global commercial model marketplace.

How does FineServe differ from proxy traces?

Proxy traces and coarse-grained generalizations fail to convey the heterogeneity of multi-model platforms. FineServe was captured "in production" and distinguishes workload regimes across three axes — model architecture, model scale, and task type — which synthetic traces do not provide.