In-Context Learning

In-context learning is the ability of a large language model to perform new tasks by reading examples or instructions placed directly in the input prompt, adapting its behavior at inference time without any update to its weights.

In-context learning (ICL) is the ability of a large language model to perform a new task by conditioning on a prompt that contains a natural-language description, a few labeled examples (shots), or both — without updating any model weights. The model produces correct outputs by pattern-matching within the context window at inference time, making it a form of meta-learning expressed entirely at the inference layer.

ICL operates through the model's attention mechanism, which allows it to read demonstrations in the prompt and infer the underlying input-output mapping. GPT-3's 2020 paper (Brown et al.) demonstrated that scaling model size dramatically amplified this emergent behavior, with zero-shot and few-shot capabilities appearing reliably only above a certain parameter threshold. The model performs no gradient descent; it encodes task information implicitly through attention over the provided examples, a process some researchers describe as implicit Bayesian inference over the prompt distribution.

ICL makes large models highly flexible without the engineering cost of fine-tuning. A developer can adapt a model to a new classification schema, output format, or domain style by writing a handful of examples into the prompt, reducing the iteration cycle from days (fine-tuning and evaluation) to minutes (prompt editing). This is especially valuable for low-resource tasks, rapid prototyping, and applications where task definitions change frequently.

As of 2026, models with context windows of 128K–1M tokens — including Claude 3.5 and 3.7 (Anthropic), Gemini 1.5 Pro (Google), and GPT-4o (OpenAI) — can accommodate hundreds of in-context examples and entire reference documents, blurring the boundary between ICL and retrieval-augmented generation. Research into why ICL works remains active, with competing accounts involving gradient descent simulation in the forward pass and task vector induction, but practical application has far outpaced theoretical consensus.