IQuest-Q1: An Open 320-Billion-Parameter MoE Model for Agentic Coding in the CLI

IQuest-Q1 is a new open model for agentic systems in the command line. It is a sparse Mixture-of-Experts with 320 billion parameters, of which 15 billion are activated for each token, or less than 5% of the weights. The authors describe a three-stage path: pretraining, mid-training, and post-training across several scaffolds. They build the final model by merging experts and checkpoints from different stages. The model is billed as strong in both coding and general tasks, and the weights are publicly available.

Pretraining: Keeping a 320-Billion-Parameter Model Stable

The first stage deals with something unglamorous but decisive: making sure a huge MoE model trains at all without blowups. The team combines three techniques. Stable SplitNorm handles training stability, the Muon optimizer handles step efficiency, and muP, a parameterization that transfers hyperparameters across scales, lets the team tune settings on small models instead of guessing on the large one.

The authors describe the main result of this combination as a wide stability margin between the safe and the optimal learning rate. That gap is usually narrow: go slightly above the optimum and the loss diverges. The wider the gap, the cheaper a tuning mistake is, and for a model this size every restart is expensive.

The curriculum has two stages. In the second pretraining stage, the share of code data in the mix increases. The model first gets a general foundation, then is shifted toward programming, which is the core skill for agents working in a terminal.

Mid-training and

Post-training: Learning from Different Scaffolds

Between pretraining and post-training sits a stage that fine-tunes the composition of the data. The mix expands across domains and formats, and the context grows gradually. For a CLI agent this is critical: it reads logs, diffs, command output, and the history of its own steps, and all of that balloons quickly.

Post-training is built around a multi-scaffold, multi-version idea. A scaffold is the harness around a model: a set of tools, a call format, an action loop. Agentic CLI products differ from one another precisely here, and a model trained for one harness often transfers poorly to another. IQuest-Q1 is trained on several at once, so that its skills do not get tied to a single interface.

Post-training combines three tools:

• SFT on demonstrations;
• domain-specific RL, where individual experts are sharpened on their own tasks;
• MOPD, which unifies signals from different training sources.

Merging Instead of a Single Best Checkpoint

The final step is model merging. The authors merge different experts and checkpoints from different stages, and work separately to keep parameter updates from conflicting. This is a known problem: two fine-tunings, each useful on its own, can cancel each other out when the weights are simply averaged. If the conflicts can be resolved, the result is a single model that keeps the strengths of the specialists, with no need to maintain a fleet of narrow versions.

Specs

• Architecture: sparse MoE, 320B parameters total, 15B active
• Pretraining: Stable SplitNorm, Muon, muP, a two-stage curriculum with a growing share of code
• Mid-training: expansion of domains and formats, gradual context increase
• Post-training: SFT, domain-specific RL, MOPD across several scaffolds and versions
• Assembly: merging of experts and checkpoints with control over update conflicts
• Licensing: open model for research and development

Who Should Look at IQuest-Q1, and Why

For teams building their own terminal agents, the value here is twofold. First, open weights for a model of this class, with 15 billion active parameters, mean paying for inference like a mid-size model rather than a dense 300-billion one. Storing the weights still takes a lot of memory, so deployment is meant for serious GPU clusters, not a single workstation.

Second, the recipe itself is interesting. Training on multiple scaffolds, merging specialized experts, and a wide learning-rate margin can be studied and reused independently of the model. The practical next step is obvious: run IQuest-Q1 on your own tasks in your own harness and compare it with your current model. Evaluation on real repositories will show more than any benchmark table.