FineServe: A Dataset of Real-World LLM Serving Workloads From a Global Marketplace

Researchers have introduced FineServe, an open dataset of real-world large language model (LLM) serving workloads. The data was collected from a global commercial marketplace and published on arXiv in July 2026 (arXiv:2607.19349). Alongside the dataset, the authors released a workload generator for benchmarking multi-model serving platforms.

What is FineServe

FineServe is an "in-the-wild" dataset of real server workloads, captured from a live commercial LLM marketplace where different models and tasks are served in parallel. Unlike synthetic data, it records how requests actually arrive and how tokens behave across heterogeneous models.

• Type: multi-model LLM serving workload dataset
• Data source: a global commercial LLM marketplace
• Publication: arXiv:2607.19349, July 2026
• Components: the dataset itself + a workload generator
• Access: open code and data on GitHub

Why proxy traces fall short

Existing LLM serving research relies on proxy traces or coarse-grained aggregates, which fail to capture the heterogeneity of modern multi-model platforms. FineServe closes this gap: the authors analyzed request arrival dynamics and token behavior along three axes at once — model architecture, model scale and task type — and found fundamentally different regimes of load fluctuation.

This matters for engineers who run LLMs as an always-on online service: delivering low latency and high throughput under volatile demand requires an accurate picture of the real traffic profile, not an averaged one.

What the workload generator does

The FineServe workload generator assembles fine-grained, "model-aware" profiles into configurable mixes for a specific benchmark scenario. In other words, a team can reproduce a load close to production and use it to run routing, scheduling and capacity planning strategies.

"By exposing these fine-grained workload dynamics,

FineServe provides a realistic foundation for evaluating routing, scheduling and capacity planning strategies in LLM serving systems," the paper's abstract on arXiv states.

According to the authors, the key value lies precisely in heterogeneity: different models and request types behave differently, and an averaged benchmark hides this.

What this means

For anyone operating multi-model platforms (and AlpinaGPT and similar aggregators are exactly this class of system), FineServe is a reason to re-check which traces their infrastructure is tested against. A realistic load profile directly affects routing choices and how many GPUs are actually needed for peak demand.

Frequently asked questions

What is FineServe?

FineServe is an open dataset of real-world LLM serving workloads plus a workload generator, presented on arXiv (arXiv:2607.19349) in July 2026. The data was collected from an operating global commercial model marketplace.

How does FineServe differ from proxy traces?

Proxy traces and coarse-grained aggregates do not capture the heterogeneity of multi-model platforms. FineServe was captured in production and distinguishes load regimes along three axes — model architecture, model scale and task type — which synthetic traces do not provide.