Stanford Posts LLM Training Lecture: 16 Chapters From Pretraining to GRPO

Stanford Online has posted the third lecture of its fall 2026 CME295 Transformers & LLMs course, and it is devoted entirely to how large language models are trained. The recording has 16 chapters, from basic pretraining to on-policy distillation. The last chapter, an end-to-end example, starts at 1:27:47, so the lecture runs well over an hour and a half. The course is taught by Stanford adjunct professors Afshine and Shervine Amidi.

The video description contains only a table of contents, with no lecture transcript or slides. So what follows is a breakdown of the material's structure and of what stands behind each term, not a retelling of what the lecturers say.

From Pretraining to Midtraining

The first half of the lecture covers the foundations. It opens with a chapter on pretraining, followed immediately by cross-entropy. This is the loss function behind next-token prediction: the less probability the model assigns to the correct continuation, the harder it is penalized.

Next come data mixtures. This is where labs decide in what proportions to blend web text, code, math and other sources. A model's capabilities depend heavily on these proportions, and labs are reluctant to publish the details.

A separate chapter covers midtraining, the intermediate stage between pretraining and post-training. It typically uses more curated data. Long-context training follows, where the model learns to work with a window much wider than in the main stage.

Post-Training: SFT, LoRA and Preferences

The second block begins with supervised finetuning. The model is finetuned on pairs of an instruction and a reference answer, which turns it from a text predictor into an assistant. LoRA comes right alongside: the base weights are frozen, and only small low-rank additions are trained. This sharply cuts memory requirements and makes it possible to finetune models on limited hardware.

The chapters on preference tuning move from theory to methods:

• the Bradley–Terry formulation, a probabilistic model of pairwise comparisons on which reward models are built;
• RLHF, where the model is optimized against a learned reward using reinforcement learning;
• DPO, which does without a separate reward model and optimizes preferences directly.

Reasoning, GRPO and Distillation

The newest block deals with reasoning models. The reasoning chapter leads to GRPO (Group Relative Policy Optimization). This is a variant of RL in which the advantage of a response is computed relative to a group of responses to the same prompt. As a result, no separate value model is needed, which makes training cheaper in memory.

The last theory chapter covers on-policy distillation. The student generates responses itself, and the teacher scores them. This yields a dense training signal without the cost of RL.

Key Details of the Lecture

• Course: CME295 Transformers & LLMs, fall semester 2026, lecture 3.
• Length: 16 chapters; the end-to-end example starts at 1:27:47.
• Pre-training topics: pretraining, cross-entropy, data mixtures, midtraining, long context.
• Post-training topics: SFT, LoRA, Bradley–Terry, RLHF, DPO.
• Advanced topics: reasoning, GRPO, on-policy distillation.
• Lecturers: Afshine Amidi and Shervine Amidi.

Who Should Watch and How

The lecture is useful for engineers who use LLMs in products and want to understand where the differences between the base, instruct and reasoning versions of the same model come from. The chapters are easy to watch by timecode. For a quick entry point, the block from 0:31:47 to 1:05:38, from SFT to DPO, is enough. If you follow reasoning models, start at 1:12:08.

The syllabus and course schedule are published at cme295.stanford.edu, and the other lectures are available in the Stanford Online playlist. If you finetune open models, map the chapters on LoRA and DPO onto your own pipeline: these are the methods most often available outside the big labs.