Amazon SageMaker Launches Full LLM Fine-Tuning: 90% Correct Code Solutions
Amazon has added full fine-tuning (FFT) to SageMaker AI. The whole model is retrained instead of small LoRA adapters, and there are no GPUs or clusters to configure. On Qwen3-4B and 110 thousand task-solution code pairs, FFT produced 90% correct solutions, as judged by Amazon Nova Pro acting as the LLM judge. If LoRA has hit a ceiling on a large dataset, you can test FFT by changing one line in the SDK. But the results come from a single task, and the blog post was written by AWS itself.
AI-processed from AWS Machine Learning Blog; edited by Hamidun News
Say you are fine-tuning a small model to write code for your own projects. The usual choice is LoRA. It leaves the main weights untouched and adds small trainable adapter matrices on top, so it needs little GPU memory.
But on hard tasks and large datasets, this approach hits a quality ceiling. Amazon now offers a way out: SageMaker AI Serverless Model Customization has gained Full Fine-Tuning (FFT), which updates all of a model's parameters. On a dataset of 110 thousand programming tasks, with 100 thousand or more training examples, FFT scored 90% on the LLM-as-Judge metric.
That means a judge model considered 90% of the solutions logically correct. You don't need to provision GPUs or a cluster, or set up distributed training.
How FFT differs from LoRA
Businesses fine-tune models for two reasons. First, the model solves a narrow task more accurately. Second, a small specialized model can match or beat a large general-purpose one on your task, at a much lower cost per token. Across millions of requests, that difference adds up.
LoRA freezes the weights of the pretrained model and adds pairs of small low-rank matrices. For simple adaptations, that is enough. But according to AWS, once the task gets harder and the examples number in the tens of thousands or more, LoRA plateaus: raising the rank, meaning the number of trainable parameters, does not close the gap with full fine-tuning. The authors conclude that the problem is not the parameter count but "rank insufficiency" itself: a low-rank update cannot capture the full complexity of the weight changes. FFT modifies the weight matrices in full, without the local distortions typical of low-rank methods.
How the test was run
AWS used the Qwen3-4B base model and the Magicoder-Evol-Instruct-110K dataset: 110 thousand instruction-response pairs covering algorithms, debugging, data structures and systems programming. The data was split 90/5/5: 99 thousand records for training, 5.5 thousand for validation and another 5.5 thousand as a held-out test set. Both methods received identical prompts, reference answers and evaluation code.
Quality was measured in three built-in ways:
- LLM-as-Judge: Amazon Nova Pro reads the prompt and the generated code and decides whether the solution is correct;
- PRIMECODE: comparison against reference code using BLEU, ROUGE-1/2/L and F1;
- MMLU: a general-knowledge test across 57 subjects, which acts as a safeguard against forgetting. Fine-tuning on code can overwrite a model's general abilities, and MMLU scores before and after show what was sacrificed.
On the radar chart, the base model is consistently weak, LoRA is noticeably better, and FFT goes further on every axis, especially BLEU and ROUGE-2. The authors say this points not to an advantage on a single metric but to a general effect of full-rank optimization.
What to do with this in practice
The service makes it quick to try many variants. In a short time, the team ran dozens of experiments on Qwen3-4B, DeepSeek-R1-1.5B, Nemotron-30B and Gemma-4-26B, varying learning rate, batch size, warmup ratio, number of epochs and dataset size. Each run took a few hours from launch to results, and the finished models landed automatically in SageMaker Model Registry as versioned packages.
Switching between methods in the Python SDK takes a single edit: in SFTTrainer, the TrainingType.FULL parameter is replaced with TrainingType.LORA, and the rest of the code stays the same. The FFT example specifies a learning rate of 2e-5, a global batch size of 8, two epochs and a warmup ratio of 0.1. These are starting values, not a universal recipe.
Learning rate matters especially in FFT because it determines how much each weight is updated. Unlike LoRA, where only small adapters change, a value that is too high in full fine-tuning irreversibly erases pretrained knowledge. So in addition to the target metric, it is worth watching MMLU.
The takeaway for readers: if you have little data or a narrow task, LoRA remains the cheaper and safer starting point. If you have a large, high-quality dataset and a complex task, and LoRA has hit its ceiling, FFT is worth testing. One caveat: the blog post was written by AWS itself, and the results come from a single task, code generation. For other domains, you will have to recheck the numbers.
- Nov 20, 2026Anthropic raises $81.3 billion in IPO, beating SpaceX's $75 billion record
- Nov 20, 2026AWS breaks responsible AI rollout for agents into 3 phases over 12 weeks
- Nov 20, 2026Anthropic raised $104.8 billion in its IPO, but its infrastructure commitments total $1 trillion
- Nov 20, 2026AWS launches managed NVIDIA Dynamo for models that don't fit on a single server
Key Questions Answered
How FFT differs from LoRA?
Businesses fine-tune models for two reasons. First, the model solves a narrow task more accurately. Second, a small specialized model can match or beat a large general-purpose one on your task, at a much lower cost per token. Across millions of requests, that difference adds up. LoRA freezes the weights of the pretrained model and adds pairs of small low-rank matrices. For simple adaptations, that is enough.
How the test was run?
AWS used the Qwen3-4B base model and the Magicoder-Evol-Instruct-110K dataset: 110 thousand instruction-response pairs covering algorithms, debugging, data structures and systems programming. The data was split 90/5/5: 99 thousand records for training, 5.5 thousand for validation and another 5.5 thousand as a held-out test set. Both methods received identical prompts, reference answers and evaluation code.
What to do with this in practice?
The service makes it quick to try many variants. In a short time, the team ran dozens of experiments on Qwen3-4B, DeepSeek-R1-1.5B, Nemotron-30B and Gemma-4-26B, varying learning rate, batch size, warmup ratio, number of epochs and dataset size. Each run took a few hours from launch to results, and the finished models landed automatically in SageMaker Model Registry as versioned packages.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.