Anthropic Circuit Tracing: How Claude Plans, Reasons, and Refuses

Anthropic circuit tracing is an unusually concrete attempt to answer a question that has haunted the generative-AI boom: when a language model produces an impressive answer, what actually happened inside the black box? Rather than treating Claude as a mysterious text generator, Anthropic’s researchers try to reconstruct parts of the computation that turn a prompt into an output.

The original video source walks through Anthropic’s March 2025 research release, On the Biology of a Large Language Model, with a useful focus on the mechanics behind the headlines. Its central subject is Claude 3.5 Haiku, Anthropic’s lightweight production model, and the company’s method for tracing internal “circuits” associated with behaviors including reasoning, multilingual translation, poem writing, refusals, hallucinations, and chain-of-thought responses. (transformer-circuits.pub)

The important takeaway is not that Claude has literal thoughts, desires, or a human-style brain. It does not. The more practical takeaway is that AI researchers are getting better at identifying internal representations, tracing how those representations influence one another, and testing whether changing them changes behavior. That could eventually make models easier to debug, safer to deploy, and more useful for teams that need to understand why an AI system acted the way it did.

Why Anthropic circuit tracing matters now

For most users, an LLM is a familiar interface: enter text, receive text. For builders, that simplicity hides a daunting engineering reality. A modern model’s output is the result of immense numbers of learned parameters and computations spread across layers, attention heads, and activations. Developers can evaluate outputs, add system prompts, fine-tune behavior, or build guardrails around a model, but they generally cannot point to an internal mechanism and say, “this is the exact process that generated that sentence.”

Anthropic frames this as a biology-like problem. The model is designed and trained by people, but many of its resulting internal strategies are not manually programmed in an understandable form. Researchers therefore need tools analogous to scientific instruments: methods that expose useful structure in a complicated system rather than pretending the entire system is already legible. Anthropic explicitly compares its goal to building a microscope for AI, capable of revealing patterns of activity and information flow that ordinary prompting cannot show. (transformer-circuits.pub)

That framing has become more relevant as language models move from novelty chatbots into search, customer support, coding workflows, enterprise knowledge systems, and autonomous-agent experiments. If a model hallucinates a policy, ignores an instruction, leaks unsafe information through a jailbreak, or provides an unfaithful explanation of its answer, output testing alone may show that something went wrong without showing why it happened.

Mechanistic interpretability is the research field aimed at narrowing that gap. It seeks explanations in terms of actual internal computations rather than only correlations between prompts and outputs. The ambition is very different from conventional benchmarking. A benchmark may establish that a model gets 82% of questions right. Mechanistic interpretability asks which internal features and pathways helped the model arrive at a particular answer—and whether intervening on those pathways predictably changes the answer.

This is why the paper deserves attention from more than AI-safety specialists. The nearer-term value is debugging. The longer-term value is governance: organizations may eventually need evidence that a model is behaving reliably for the reasons they expect, especially in high-stakes workflows.

The core problem: neurons are not understandable concepts

A natural instinct is to inspect the model’s neurons directly. Unfortunately, this does not work as neatly as it might in a simple neural network diagram. Individual model neurons can be polysemantic, meaning a single neuron can participate in several unrelated concepts or patterns. At the same time, one useful concept may be distributed across many neurons.

This is connected to a widely studied idea called superposition. Models have pressure to represent more concepts than their available neurons can cleanly dedicate one-to-one, so concepts may be encoded as overlapping patterns rather than discrete switches. That makes raw activations difficult to interpret: a high-value neuron does not necessarily mean “the model is thinking about invoices,” “the model recognizes French,” or “the model is planning a rhyme.” (transformer-circuits.pub)

Anthropic’s earlier dictionary-learning work provided part of the answer. Rather than treating individual neurons as the meaningful unit, researchers identify recurring patterns across activations and call them features. A feature is a learned direction or pattern that may correspond to a recognizable concept, structure, token pattern, or behavior-relevant abstraction.

That does not mean every feature has a clean English description. Some may reflect narrow syntactic patterns, mixtures of related contexts, or representations humans have not named. But the approach can make the model’s internal state more sparse and inspectable. Anthropic previously reported identifying millions of such concept-like features in Claude Sonnet, while emphasizing that raw neural activity is otherwise just a long list of numbers without obvious semantic meaning. (anthropic.com)

Features are an essential starting point, but they are insufficient on their own. A list of active concepts does not explain a behavior any more than a list of active brain regions explains how someone solved a math problem. Researchers also need to understand relationships: which features influence other features, at what token positions, and which paths contribute to a particular next-token prediction.

That is the gap Anthropic circuit tracing tries to fill.

How circuit tracing works

At a high level, Anthropic circuit tracing constructs an interpretable approximation of a model’s computations and uses it to build a graph of influential internal pathways. The work does not claim to fully decode every computation in Claude. Instead, it attempts to recover a meaningful, testable portion of the computation relevant to a specific prompt and output.

The method’s key component is a replacement model. In the research setup, Anthropic replaces the transformer’s multi-layer perceptron, or MLP, components with a more interpretable structure called a cross-layer transcoder. The transcoder is trained to approximate the original model’s MLP behavior while representing activity through sparse features. (transformer-circuits.pub)

Why use a replacement model?

The replacement model is a compromise. It is not the original network, so it creates an obvious scientific risk: perhaps the more interpretable surrogate tells a cleaner but misleading story. Yet the original MLP activations are hard to turn into human-comprehensible mechanisms because of polysemanticity and overlapping representations.

A cross-layer transcoder is designed to make causally relevant routes easier to inspect. Instead of only relying on the immediately preceding layer in the same way as a conventional feed-forward stack, it can receive information from earlier layers. This gives researchers a more direct view of where a feature’s contribution may originate. Sparse activation also encourages fewer features to be active for a given computation, making an attribution graph less like an indecipherable cloud of numbers.

The method paper describes the result as a graph description of computation on prompts of interest. Nodes correspond to sparse features, while edges represent attributions: estimated influence from a source feature to a later target feature. Anthropic used visualization and validation tools on an 18-layer language model before applying the framework to Claude 3.5 Haiku. (transformer-circuits.pub)

What an attribution graph can show

An attribution graph begins with a concrete question. Suppose a prompt introduces a multi-word organization name and then begins an acronym in parentheses. Why does the model predict the next acronym letter? Or suppose a prompt asks for a simple chained inference. Which internal representations help the model retrieve the first fact, materialize an intermediate conclusion, and reach the final answer?

Instead of producing one global explanation for the entire model, the graph highlights features that appear to matter for that particular forward pass. It can include token-level cues, lexical patterns, abstract concepts, candidate output words, and downstream effects on the logits that make one next token more likely than another.

This specificity is a strength and a limitation. It is a strength because AI failures are often contextual. The same model can correctly refuse a harmful request in one phrasing and behave inconsistently in another. It is a limitation because a diagram for one prompt is not a complete theory of a capability across every prompt, language, or deployment setting.

The decisive test: intervention, not storytelling

The most compelling part of the research is not the graph visualization itself. It is the use of interventions.

Interpretability work can easily drift into persuasive storytelling: researchers inspect a cluster of activations, attach a plausible label, and create a narrative about what the model “must be doing.” Anthropic’s approach attempts to move beyond that by altering relevant internal features and observing whether the model’s behavior changes in the predicted direction.

If a feature is said to support a particular concept or output, suppressing it should reduce that contribution. If a feature is said to promote another concept, artificially increasing or injecting it should raise the likelihood of related outputs. When these interventions have predictable effects, the feature labels and graph edges gain causal credibility rather than remaining purely descriptive.

This distinction matters for anyone evaluating AI claims. An activation that appears correlated with a topic is not necessarily a cause of an answer. Intervention asks the stronger question: if we change this mechanism, does the downstream behavior move as the hypothesis predicts?

Anthropic’s broader framing is appropriately cautious. The research describes partial pathways and evidence, not complete access to a model’s inner life. The company also acknowledges limits to inferring too much from a model’s visible chain of thought, since a written explanation can be a plausible post-hoc account rather than a faithful account of the model’s full internal computation. (anthropic.com)

For product teams, this creates a useful standard: do not treat model-generated rationales as audits. A chain-of-thought-style answer may be helpful to a user, but it is not automatically a faithful telemetry log. If dependable explanations matter, organizations will need evaluation methods, controlled tests, and—eventually—mechanistic evidence that is more robust than self-reporting text.

What the research found about multi-step reasoning

One of the paper’s most accessible examples concerns multi-step reasoning. A language model can answer a question that requires more than one fact: for example, determine an intermediate relationship and use it to infer a final answer. The interesting question is whether the model internally represents the intermediate fact or jumps directly from a familiar prompt pattern to a likely answer.

Anthropic’s analysis suggests that Claude 3.5 Haiku can materially represent intermediate conclusions during these tasks. In other words, the model’s path is not always reducible to a single shallow association between the input and final answer. The attribution graphs identify internal features associated with intermediate facts that then contribute to the final response. (transformer-circuits.pub)

That is significant, but it should not be overread. The finding does not prove that the model reasons exactly as a person does, nor does it imply every apparently logical answer comes from a rigorous multi-step procedure. Language models can use mixtures of strategies. On some prompts, they may route through intermediate representations; on others, strong associations or familiar templates may dominate.

Why this distinction matters to builders

For an AI application, the relevant question is not “does the model reason?” in the abstract. It is: under which conditions does it use a reliable reasoning path, and when does it fall back on a shortcut?

Consider a legal-document assistant, a financial-analysis copilot, or a support agent with a retrieval layer. If it produces the correct answer because it genuinely integrated the supplied documents, that is different from producing the correct answer because a common phrase triggered a memorized response. The latter can fail catastrophically when a prompt is reworded, facts are updated, or the question lies outside the training distribution.

Circuit-level evidence may eventually help distinguish those cases. It could identify whether a model incorporated retrieved information, tracked a premise across several steps, or prematurely activated a high-confidence answer pattern. That is not a finished product capability today, but it is a valuable direction for evaluation research.

What poem planning reveals about next-token models

The poem experiment is likely the paper’s most intuitive result. Language models generate text sequentially, one token at a time. That can lead users to assume they merely improvise locally: predict the next word, then repeat. Yet competent poetry often requires a longer horizon. A poet writing a rhyming couplet must choose phrasing for the current line that leaves a viable route to a target rhyme later.

Anthropic’s circuit tracing analysis found evidence that Claude activates representations of possible rhyme targets before it reaches the final word of a line. In the company’s interpretation, the model can plan several words ahead and write toward a destination rather than discovering the rhyme only at the last moment. (anthropic.com)

The lesson is broader than poetry. An autoregressive interface does not prove a purely myopic internal process. A model still outputs tokens one at a time, but the computations that generate the current token can represent future-oriented constraints: a rhyme, an expected answer format, a code structure, a multi-part argument, or an anticipated translation choice.

For creators, that helps explain why prompt structure matters so much. Asking an AI to “write a catchy launch poem” produces a very different internal challenge from requesting a specific syllable count, rhyme scheme, product detail, voice, and ending. Constraints can give a model a clearer long-range target. But constraints also compete: more instructions may help planning, or they may overload the context and create conflicts that the model resolves unpredictably.

A practical prompting implication

When a deliverable requires coherence across many sentences, do not rely on a single vague generation request. Give the model an explicit destination and a structure for reaching it.

• Specify the final artifact’s format before asking for prose.
• State non-negotiable elements that must appear near the end, not only at the beginning.
• Ask for an outline, message hierarchy, or checklist before the final draft when accuracy matters.
• Use a separate review pass to verify that the output achieved the planned constraints.

These techniques do not expose internal features. They simply make it easier for the model to organize around a longer-horizon objective—the same general phenomenon that the rhyme analysis makes visible.

Multilingual representations are more conceptual than expected

Anthropic also investigated multilingual processing and reported evidence that Claude sometimes uses a shared conceptual space across languages. When translating simple sentences into multiple languages, the researchers observed overlap in internal processing, which they interpret as support for a kind of language-independent representation of meaning. (anthropic.com)

That does not mean Claude has discovered a pristine universal language hidden behind human languages. Nor does it erase real differences among languages, cultures, writing systems, and translation conventions. The research instead suggests that, for some tasks, internal representations may be organized around concepts that can be reused across different linguistic surface forms.

This has implications for international marketing and product design. Many teams assume that an AI’s ability to translate naturally means it understands every market nuance. The circuit-tracing result suggests a more subtle picture: models may share semantic machinery across languages, which can be powerful for transfer and consistency, while still failing on region-specific context, idioms, regulatory terminology, or audience expectations.

The practical recommendation is to separate semantic transfer from localization quality. Use AI to accelerate first drafts, terminology extraction, message mapping, and cross-market content adaptation. But retain native-language review for claims, tone, compliance, and cultural relevance. A shared conceptual representation may make multilingual AI capable; it does not make local expertise optional.

Safety research: refusals and jailbreaks are circuits too

The paper also examines refusals, jailbreak behavior, and hidden-goal-style scenarios. These examples are important because safety training is often assessed from the outside: a red-team prompt succeeds or fails, a refusal is judged adequate or inadequate, and researchers tally outcomes. Circuit tracing adds a possible internal layer of diagnosis.

A model may refuse a request because it recognized a harmful concept, because a formatting cue activated a safety response, because it inferred policy context, or because another competing behavior won out. Those routes can matter. Two prompts with identical visible refusals may rely on different internal mechanisms, and one mechanism may be more fragile under adversarial paraphrasing.

Anthropic’s April 2025 follow-up illustrates the point. While examining an unsuccessful jailbreak involving a weapon-related request, the team found that the model appeared to refuse it for reasons different from the baseline direct request. The researchers also found examples where narrow feature visualizations could be misleading, reinforcing the need for diverse examples and careful validation rather than overconfident labels. (transformer-circuits.pub)

That is a healthy warning for the field. Interpretability should not become a new source of false certainty. A colorful graph and a human-readable feature name are not proof that researchers have found the full mechanism. The value comes from repeated testing, alternative prompts, interventions, replication, and explicit disclosure of uncertainty.

For founders building on third-party models, the operational message is straightforward: safety cannot be delegated entirely to the base model’s refusal behavior. Use layered defenses, including scoped tools, retrieval permissions, output checks, audit logs, rate limits, human escalation, and threat-model-specific testing. Mechanistic interpretability may eventually improve the model layer, but application security remains an application responsibility.

The biggest limitation: the graphs were incomplete

The video source correctly emphasizes the central methodological challenge: is the transcoder showing the real transformer mechanism, or a cleaner approximation that reaches similar conclusions by a different path?

Anthropic addresses this with approximation checks and interventions, but the issue remains fundamental. A replacement model is useful precisely because it is easier to analyze than the original. That does not guarantee perfect faithfulness. Researchers must continually ask whether an apparent circuit exists in the original model, whether the graph omitted important components, and whether interventions generalize outside the narrow prompt studied.

There was also a specific incompleteness in the original attribution-graph approach. The initial work made significant progress on MLP computations but did not fully explain why attention heads selected particular context positions. In July 2025, Anthropic introduced “QK attributions” to trace feature interactions that shape attention scores, explicitly describing this as a way to fill a major missing piece in earlier attribution graphs. (transformer-circuits.pub)

That update matters because attention is not a peripheral detail. It is how transformer models move information across positions in the context. If a model needs to find the right clause in a long contract, associate a pronoun with an earlier entity, or retrieve a key detail in a prompt, attention behavior can be central to the explanation.

The evolving method should therefore be viewed as a research program, not a solved decoder ring. Anthropic’s own later updates demonstrate that the team continues to identify blind spots, improve tooling, and revise earlier assumptions. That is a sign of scientific maturity, not a reason to dismiss the work.

How this compares with ordinary AI evaluation

Circuit tracing does not replace benchmarks, red teaming, human evaluation, or production monitoring. It answers a different class of question.

The strongest AI assurance programs will combine all of them. A benchmark can reveal a performance regression. Production logs can show when it reaches users. Human experts can assess severity. A mechanistic tool may eventually help locate the internal pathway that created the regression and test an intervention that reduces it without breaking unrelated capabilities.

This is especially relevant for teams tempted to choose between “black-box evaluation” and “interpretability.” That is the wrong choice. Interpretability is not a substitute for outcome measurement; it is an additional diagnostic layer.

What creators, marketers, and founders should do with this research

It would be easy to treat Anthropic circuit tracing as fascinating but irrelevant academic work. That would miss its practical signal: AI products are becoming sophisticated enough that output-only thinking is not always sufficient.

Creators should take the poem-planning result as evidence that models can work toward structural goals, but should still use planning prompts and verification passes for high-value content. Marketers should treat multilingual ability as leverage for semantic adaptation, not a replacement for local review. Founders should assume that model behavior can shift for subtle reasons and build systems that measure failure modes rather than relying on demos.

Here is a pragmatic playbook:

1. Define failure modes before scaling usage. List the outcomes that would damage users or the business: invented facts, harmful recommendations, missed instructions, privacy leaks, unsupported claims, or inconsistent brand voice.
2. Create scenario-based evaluations. Test normal prompts, adversarial paraphrases, long-context cases, ambiguous instructions, and inputs that resemble real customer traffic.
3. Separate generation from verification. Let the model draft, classify, extract, or summarize—but add validation rules, retrieval citations, structured outputs, or human approval where the cost of error is high.
4. Log enough context to investigate failures. Capture prompts, model versions, tools used, retrieved documents, output, reviewer decisions, and downstream outcomes within appropriate privacy boundaries.
5. Watch interpretability developments without waiting for magic. Circuit-level tools are promising, but today’s production safety controls still need conventional engineering discipline.

The business implication is not that every startup needs an interpretability research team. It is that trustworthy AI products will increasingly differentiate themselves through observability, evaluation rigor, and honest limits—not merely through access to the most capable base model.

The broader AI biology moment

Anthropic’s “biology” metaphor has spread because it captures an uncomfortable reality. These systems are engineered, yet their learned internal mechanisms can be difficult for their creators to explain. Researchers increasingly study them through a blend of theory, probing, experiments, ablations, and causal interventions—methods that resemble experimental science as much as conventional software debugging.

That work is no longer confined to one company. MIT Technology Review recently described mechanistic interpretability as an emerging effort across organizations including Anthropic, OpenAI, Google DeepMind, and independent research groups, driven by the challenge of understanding systems whose scale exceeds direct human comprehension. (technologyreview.com)

Still, the field should resist anthropomorphic shortcuts. Terms such as “thinking,” “planning,” and “goals” are convenient descriptions of observed computational behavior, but they can mislead if readers interpret them as claims about consciousness or human-like intent. The useful question is more precise: what representations, routes, and interventions predict a model’s output under defined conditions?

Anthropic’s paper is valuable because it pursues that question with more rigor than a generic visualization exercise. It identifies features, maps pathways, and tests selected claims through intervention. It also exposes the unfinished nature of the task: attention remained a major gap in the early approach, feature labels can mislead, and circuit explanations must be continuously validated.

Conclusion: from impressive outputs to inspectable systems

Anthropic circuit tracing will not make Claude—or any frontier model—fully transparent overnight. Modern LLMs remain vast, distributed systems whose behavior cannot be reduced to a few tidy diagrams. But the research changes the direction of travel.

Instead of asking only whether an AI system can write a poem, solve a reasoning prompt, translate a sentence, or reject a jailbreak, researchers can increasingly ask what internal process supported that outcome. The answer may reveal genuine planning, a useful intermediate representation, a brittle shortcut, or an unexpected safety pathway.

For people building with AI, that is the most important idea in the paper. Capability matters. But as models are trusted with more consequential work, the ability to inspect, test, and eventually modify behavior will matter too. The next phase of AI competition may not be only about who can produce the strongest answer. It may also be about who can provide the strongest evidence for why the answer appeared.

FAQ

What is Anthropic circuit tracing?

Anthropic circuit tracing is a mechanistic-interpretability method that uses a more interpretable replacement model to map feature-level computational paths associated with a language model’s output. The resulting attribution graphs aim to show how internal features influence later features and predicted tokens. (transformer-circuits.pub)

What did the Claude 3.5 Haiku study find?

Anthropic reported evidence that Claude 3.5 Haiku can represent intermediate facts in some multi-step reasoning tasks, plan rhyme targets ahead during poem generation, and use overlapping conceptual representations when processing multiple languages. (transformer-circuits.pub)

Does circuit tracing prove that AI thinks like a human?

No. The research analyzes computational mechanisms and causal influences inside a model. Words such as “planning” describe observed functional behavior; they do not establish consciousness, subjective experience, or human-style cognition.

Why are intervention experiments important?

They help distinguish causal mechanisms from appealing stories. If suppressing or increasing a feature changes model behavior as predicted, researchers have stronger evidence that the feature and its connections genuinely contribute to the output.

Can businesses use circuit tracing to make AI safe today?

Not as a turnkey safety product. The research is promising, but partial and technically demanding. Businesses should continue using layered controls such as evaluations, access restrictions, retrieval validation, output checks, monitoring, and human review while interpretability tools mature.