Synthetic Data for Rare Languages: Boosting Julia Code Generation by 14 Points

Researchers have developed Selective Left-Shift, a method that lets small language models generate quality code in rare programming languages such as Julia and Ballerina. Testing the approach on Qwen3-8B, they raised accuracy on Julia by 14.2 points while using just a third of the training data and a sixth of the compute required by previous approaches.

Why rare languages are hard for models

Modern LLMs generate Python and Java code well, since the internet holds millions of examples of these languages. For rare languages (Julia, Ballerina, Nim), performance drops sharply. Trying to improve the situation with small models runs into three obstacles.

• Too little training data for the specific syntax of a rare language
• High cost of scaling compute at code-generation time
• Low efficiency of reinforcement learning from scratch

How Selective Left-Shift works

Instead of increasing compute at generation time, the researchers "shifted" it to the left, into the preparation of training data. The three-phase pipeline works as follows:

Phase 1: Synthesis with verification. The model generates code in the rare language, and a compiler and tests check whether the solution works. Iterative feedback improves the examples.

Phase 2: Fine-tuning on synthetic data. Qwen3-8B trains on the verifiable examples, building the rare language's syntax into its representations.

Phase 3: RL with verifiable rewards. The reward is based on language-independent input-output tests. Fine-tuning restricts the search to syntactically correct variants, which makes training more stable.

How much accuracy improved

On the MultiPL-E benchmark for Julia, the improvement was +7.6 points pass@1; on LiveCodeBench, +14.2 points. The main achievement is resource savings.

• 3 times less training data than before
• Compute costs fell 6-fold
• The method generalizes to Ballerina, a language absent from training, reaching 49.7% pass@1

What this means for developers

The study shows an effective way to work with rare languages: it is better to invest resources in high-quality synthesis and verification of training data than in scaling the model or compute. This is useful for in-house and young programming languages, where little public code exists on the internet.