Activation Steering: A Tutorial on Controlling a Language Model from the Inside with PyTorch and nnsight

A tutorial from Habr explains Activation Steering, a method for controlling a language model without retraining. It covers three approaches, working Python code, and a demonstration: a deliberate shift of the model toward toxic responses, meant to show how precise the intervention can be.

What Is Activation Steering

Activation Steering lets you control a language model's behavior without changing its weights and without running fine-tuning. During inference, the researcher intercepts the network's internal activations at a chosen layer and adds a directional vector to them. The result: the model starts generating text with the specified property.

The method builds on one of the key discoveries in mechanistic interpretability: the activation space of an LLM turns out to be structured. Different concepts, such as anger, politeness, confidence, conversation topic or language, are encoded as relatively linear directions in this high-dimensional space. Finding the right vector means getting a direct control lever without altering the weights.

A steering vector is obtained by a contrastive method: take examples with the desired property and without it, run both sets through the model, and compute the difference of the mean activations. The resulting vector is added to the activations of the chosen layer with a scaling coefficient.

Three Implementation Approaches

The tutorial covers three tools with increasing levels of abstraction:

• pytorch-hooks: `register_forward_hook` intercepts the activation tensor of the chosen layer, the vector is added, and the modified tensor is returned to the computation graph. Maximum control, minimal dependencies.
• nnsight: a library with declarative syntax. The intervention code reads almost like straightforward pseudocode, which suits experiments in a Jupyter notebook.
• pyvene: a high-level framework for causal interpretability. It supports reproducible experiments and easy switching between transformer layers.

The choice depends on the task: pytorch-hooks when you need full control; nnsight for readable research code; pyvene for structured causal analysis.

Where Steering Is Used

The tutorial's demo shifts a model toward hate speech. The choice is deliberately uncomfortable: it shows plainly that the intervention works. The same tools are also used to detect and neutralize unwanted behavior, so steering works in both directions.

Practical applications:

• Alignment research: study which concepts are encoded in the network and how separable they are
• Safety red-teaming: test whether unwanted behavior can be activated without training data
• Interpretability: determine which transformer layers are responsible for specific semantic properties
• Editing without fine-tuning: remove or amplify a pattern with a targeted intervention

What This Means

A few years ago, Activation Steering was a tool of academic labs, used by researchers at Anthropic, DeepMind and EleutherAI in mechanistic interpretability work. The arrival of nnsight and pyvene lowered the barrier to entry to the level of ordinary PyTorch code.

A Russian-language tutorial on Habr is a rare case of such a specialized topic getting a high-quality explanation without a language barrier. For teams working on the safety and alignment of language models, mastering steering is becoming a practical skill rather than an academic exercise.