Avito Breaks Down Six KubeCon Japan 2026 Talks on Sharing GPUs

Anton Alekseev, an ML Ops engineer at Avito, traveled to Yokohama for KubeCon + CloudNativeCon Japan 2026 and pulled six talks on GPUs and LLM inference into a single write-up. By the second day of the conference, he noticed that the slides from different speakers looked alike: the tenant isolation spectrum, quotas, bin packing, and the refrain "GPUs are expensive, so let's share them." Avito is solving the same problem, so the comparison turned out to be practical: what the company has already tried and what it is only planning to do.

The Habr article itself currently consists only of an introductory section, so what follows covers the context rather than retelling each talk.

One theme across six talks

The fact that different teams independently arrive at the same patterns says something about how mature the topic is. A couple of years ago, GPUs in Kubernetes were usually handed out whole: one card, one pod. Today accelerators take up a noticeable share of the infrastructure budget, and their load is uneven. Training and batch jobs load a card heavily. Online inference often leaves it half empty. That gives four recurring themes:

• tenant isolation: how strictly to separate neighbors on the same card;
• quotas: how many accelerators a team or project may occupy;
• bin packing: how densely to pack pods onto nodes so that partially used cards don't multiply;
• LLM inference as the main consumer, where memory for weights and the KV-cache runs up against the card's size.

The isolation spectrum

This spectrum is described by a single scale: the harder the boundary between tenants, the lower the utilization and the higher the predictability. The typical points on it have been known for a long time.

• A whole card per pod: maximum isolation, lowest density.
• MIG on supported NVIDIA cards: hardware partitioning into several independent instances, up to seven per card, with dedicated memory.
• MPS: several processes share compute resources, but isolation is weaker.
• Time-slicing: pods take turns on the card, and nobody guarantees memory or isolation from errors.

The choice depends on the workload. For a critical online service with strict latency requirements, a hardware boundary makes more sense. For internal experiments, soft partitioning is usually enough.

Quotas and packing

Isolation settles the question of who sits next to whom. Quotas and the scheduler decide who gets a card in the first place. In practice, this is where batch job queues, priorities, and preemption come together. Bin packing matters because fractional requests quickly fragment a cluster: there are plenty of free slices, but nowhere to place a pod that needs a whole card or a large MIG profile. A topology-aware scheduler that aims to fill nodes densely puts those slices back into use.

According to the author, he will go through which of the presented ideas Avito has already tried and which are still planned. For the reader, this is the most valuable part: conference talks usually show a working setup but rarely explain where it breaks on someone else's workload. A report from an engineer who checks the slides against his own production fills exactly that gap. At the end of the text, the author promises a story about Japan itself, so the format is closer to a travel diary with a technical core.

Where a team with a GPU cluster should start

If you are only thinking about sharing cards, the order of steps follows naturally from the talks themselves. First, measure actual utilization: compute-unit load and memory in use for each service, not just the fact that a card has been allocated. Then split workloads by isolation requirements: online inference, batch jobs, experiments. Pick a separate point on the spectrum for each class instead of trying to impose a single regime. After that, introduce per-team quotas and queues for batch jobs, and tune packing last, once the structure of requests is clear. Finally, decide in advance what you will do about noisy neighbors: without latency metrics, sharing a card quickly turns into a source of unexplained degradation.

The full breakdown of the six talks and Avito's experience are published on Habr in Anton Alekseev's article.