Синтез данных для редких языков: как улучшить код-генерацию на Julia в 14 раз

Researchers have developed the Selective Left-Shift method, enabling small language models to generate high-quality code in rare programming languages like Julia and Ballerina. Testing the approach on the Qwen3-8B model, they increased Julia accuracy by 14.2 percentage points, using only one-third of the training data and one-sixth of the computational resources of previous approaches.

Why rare languages are challenging for models

Modern LLMs generate code well in Python and Java — there are millions of examples of these languages on the internet. For rare languages (Julia, Ballerina, Nim), performance drops sharply. Attempting to improve the situation using small models faces a triple barrier.

• Lack of training data specific to the syntax of rare languages
• High computational costs during scaling when generating code
• Low efficiency of reinforcement learning from scratch

How Selective Left-Shift works

Instead of increasing computational resources during generation, researchers "shifted" them left — into training data preparation. The three-phase pipeline works like this:

Phase 1: Synthesis with verification. The model generates code in a rare language; compiler and tests check whether the solution works. Iterative feedback improves examples.

Phase 2: Fine-tuning on synthetic data. Qwen3-8B is trained on verifiable examples, embedding the syntax of the rare language into its representations.

Phase 3: RL with verifiable rewards. Reward is based on language-independent input-output tests. Fine-tuning constrains the search to syntactically correct variants, making learning more stable.

How much accuracy improved

On the MultiPL-E benchmark for Julia, the improvement was +7.6 percentage points pass@1; on LiveCodeBench — +14.2 percentage points. The main achievement is resource savings.

• Training data usage reduced by 3 times compared to before
• Computational costs reduced by 6 times
• Method generalizes to Ballerina, a language not present in training — 49.7% pass@1

What this means for developers

The research shows an efficient way to work with rare languages: it is better to invest resources in high-quality synthesis and verification of training data than in scaling the model or computational resources. This is useful for internal and emerging programming languages where there is little public code on the internet.