Avito Built Aviflow: An ML Platform for Distributed LLM Training on HGX

Avito has gone from dedicated machines with SSH access to Aviflow, its own cloud-native platform for training large language models. Pavel, a senior developer on Avito's ML platform, has worked in MLOps for a year and a half. He gave a talk at Kuber Conf and reworked it into an article on Habr. The title promises "LLM training in one click" on HGX servers. The visible part of the material cuts off at the history of the platform, so there are no concrete figures yet on the number of nodes and GPUs. It does, however, describe well the evolution that many companies with a growing ML team go through.

How it started: DevBox and SSH

When Avito had only a few data scientists, each of them worked with their own set of tools. The main one was DevBox, a dedicated machine. A specialist logged in over SSH, assembled datasets, trained models and did the rest of the work by hand. While the team is small, this setup is tolerable. The machine is assigned to one person, there are no conflicts over resources, and a single admin fixes any problem.

Then the number of specialists and departments grew. Colleagues locked entire machines and ran jobs manually. GPUs sat idle between runs, and the queue turned into correspondence in chats.

LLMs added chaos

On top of this, the company added LLMs. According to the author, LLMs come with their own tooling and did not make life any easier. The reason is clear from the nature of the task. A model does not fit on a single card or takes too long to train, so it needs several GPUs, often across several nodes. At that point DevBox stops working as a unit of resource. The team needs a scheduler, environment isolation, reproducible runs and clear reporting on utilization.

Avito's answer is called Aviflow. It is a cloud-native platform where training is described as a job, not as a set of manual actions on a specific machine.

Why HGX and Kubernetes

HGX is NVIDIA's reference server platform, usually with eight GPUs connected by the high-speed NVLink bus through NVSwitch. Connectivity is what matters for distributed training. Gradients have to be synchronized quickly, or the GPUs end up waiting for each other. The talk was given at Kuber Conf, which means orchestration is built around Kubernetes, and HGX servers become a resource pool rather than personal machines.

Key facts from the available part of the material:

• Author: Pavel, senior developer on Avito's ML platform, a year and a half in MLOps.
• Platform: Aviflow, cloud-native.
• Predecessor: DevBox, dedicated machines with SSH access.
• Hardware: HGX servers.
• Task: distributed LLM training.
• Format: a talk at Kuber Conf and a write-up on Habr; a video recording is available.

What you can take from this

Avito's story shows a typical path, and you can use it to check your own team against it. If specialists negotiate GPU access in a messenger, you already need a platform. If launching training requires SSH, it is time to move to describing jobs declaratively. If LLMs have appeared in your team as a separate island, decide in advance how their jobs will live in the shared cluster.

The practical order of steps is this: first a shared GPU pool and a queue, then standardized environments, then distributed launches and monitoring. Starting with the "button" is pointless if there is no resource accounting behind it.

For details, including Aviflow's architecture, scheduling design and results, read the full text on Habr or watch the recording of the talk. That is where the figures on utilization and launch speed should be, since the announcement part does not include them.