Anthropic opens a window into Claude's thinking: the Jacobian lens catches hidden concepts
Anthropic has introduced the Jacobian lens, an interpretability tool that opens up the LLM black box. It shows which concepts activate in Claude's intermediate layers when it answers a question. The findings range from the mundane to the unusual: the model builds hidden representations of ideas that do not appear in its ordinary output. This offers a first, naive look at how LLMs "think."
AI-processed from MIT Technology Review; edited by Hamidun News
Anthropic has developed a technique called the Jacobian lens, which offers the clearest view yet of what happens inside Claude when it answers questions or performs tasks.
What it finds
When Claude answers a question, the Jacobian lens shows activations in the intermediate layers. The results range from the everyday (a layer tied to numbers activating while solving a math problem) to the unusual (hidden concepts that do not appear in the explicit output).
How it works
Instead of inspecting weights and activations blindly, the tool uses Jacobian mathematics (a matrix of derivatives) to track how changes in the input affect intermediate representations. This makes it possible to:
- See which abstract concepts the model activates
- Trace the path of logic from input to output
- Understand which faulty chains of reasoning lead to wrong answers
Why it matters for safety
Uncovering hidden concepts matters for AI safety: if we see that a model activates dangerous concepts, even when the final answer is safe, we can fix it.
- Nov 1, 2026OpenAI Rewrites GPT Behavior Rules: Five Values and a Chain of Command
- Nov 1, 2026Google admits AI Mode cuts search revenue, shares fall 15%
- Oct 31, 2026Author puts his script before an AI jury: the verdict cost 850 thousand tokens
- Oct 31, 2026Anthropic: Fold Subagents Back Into a Single Loop — One Customer's Tasks Ran Twice as Fast
Key Questions Answered
What it finds?
When Claude answers a question, the Jacobian lens shows activations in the intermediate layers. The results range from the everyday (a layer tied to numbers activating while solving a math problem) to the unusual (hidden concepts that do not appear in the explicit output).
How it works?
Instead of inspecting weights and activations blindly, the tool uses Jacobian mathematics (a matrix of derivatives) to track how changes in the input affect intermediate representations. This makes it possible to: See which abstract concepts the model activates Trace the path of logic from input to output Understand which faulty chains of reasoning lead to wrong answers
Why it matters for safety?
Uncovering hidden concepts matters for AI safety: if we see that a model activates dangerous concepts, even when the final answer is safe, we can fix it.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.