NVIDIA Open-SWE-Traces: Building a Dataset to Fine-Tune Coding Agents

NVIDIA has released the Open-SWE-Traces dataset, a collection of real multi-turn sessions in which AI agents solve software development tasks. The tutorial covers the full pipeline, from streaming the data to a finished set for supervised fine-tuning.

What Open-SWE-Traces is

The dataset contains thousands of agent trajectories. Each one is a full record of a session in which the AI receives an engineering task, calls tools step by step (reading files, running tests, searching code), iterates on a solution and produces a final patch. This differs fundamentally from ordinary question-answer datasets: it records not only what worked, but how the agent got there.

Each record contains structured metadata:

• trajectory length: the number of agent steps
• the list of tools used and how often they were called
• the size of the final diff in lines of code
• the programming language of the task
• a flag for a successful or unsuccessful solution

The data is hosted on Hugging Face and supports streaming. In Google Colab you can work with the dataset without downloading it in full, which matters at sizes of several gigabytes.

How the pipeline is built

The tutorial walks through several processing stages.

First is dialogue normalization. Multi-turn agent sessions are converted to a single format: user messages, agent replies and tool calls are aligned into one sequence. This is needed because different agent versions log sessions differently.

Second is patch parsing. The code changes themselves are extracted from the agent's final output in unified diff format. This patch becomes the "answer" in the training example.

Third is building an analytical DataFrame. Key metrics are computed for each trajectory: token budgets at different stages of the agent's work, distribution across tools, and success statistics by language and task type.

Filtering for SFT

The final step is selecting training examples. The authors apply a chain of filters.

By success label: only trajectories with a successful solution make it into the set. Training on failed sessions without special annotation is risky: the model will absorb faulty patterns.

By tokens: trajectories longer than a set limit are dropped. Overly long examples do not fit into the context at standard training settings.

By language: if you need a specialized agent for Python or JavaScript, filtering keeps only the relevant examples.

By presence of a patch: sessions without final code are useless for SFT, where the model must learn to produce a concrete result.

"Training data quality matters more than quantity, especially for

agent traces, where failed sessions can lock bad patterns into the model."

What this means

NVIDIA's Open-SWE-Traces is one of the first public datasets of real agent trajectories for engineering tasks. The tutorial provides a working template: from raw data on Hugging Face to a ready SFT set in a few lines of code. For teams building their own coding agents, it is a ready starting point that removes the need to collect data from scratch.