AI Coding Agent Observability: Why Rungraph Makes Agent Runs Reviewable

AI coding agent observability is quickly becoming a practical engineering problem, not a niche concern. As Claude Code, Codex, and similar tools produce more code with less hands-on typing, the hardest question is shifting from “can the agent build this?” to “what exactly did the agent do, and can we verify it?”

A new open-source project called Rungraph is built around that shift. First shared in a Reddit post in r/SaaS, Rungraph reads coding-agent session records already stored locally, reconstructs them as an interactive graph, and lets developers query the run through Model Context Protocol (MCP) tools. The core promise is straightforward: instead of treating an agent’s terminal output as an unreadable wall of text, teams can inspect its decisions, tool calls, retries, subagents, failures, tests, and human interventions as connected evidence.

That idea matters because agentic coding is becoming more autonomous. Anthropic made Auto mode the default starting mode for new Claude Code sessions on Pro, Max, and Team plans on August 14, 2026, reducing approval friction for longer-running tasks. Greater autonomy can improve throughput, but it also raises the value of clear after-the-fact inspection. (claude.com)

The new bottleneck in AI-assisted development

Early AI coding workflows were constrained by generation quality. Developers spent much of their time prompting, correcting suggestions, and manually converting snippets into working code. Coding agents changed the shape of the work: they can inspect repositories, edit multiple files, run commands, execute tests, delegate portions of a task, and loop through failures without waiting for a developer to copy and paste each instruction.

The result is not simply “faster coding.” It is a growing gap between the speed of agent execution and the speed of human comprehension.

A typical coding-agent session can include:

• A user request and a plan.
• Multiple repository searches and file reads.
• Shell commands, package installs, and test runs.
• File edits across a broad diff.
• Failed commands followed by retries or alternative approaches.
• Permission denials, clarifying questions, or mid-run human corrections.
• Subagents that investigate, implement, review, or test in parallel.

A final pull request shows the code that survived. It does not necessarily show the path taken to get there.

That distinction is important. A diff can be clean while the execution path was wasteful, risky, or based on a faulty assumption. Conversely, a messy-looking sequence of retries may reflect reasonable debugging work that ends in a correct, well-tested change. Traditional code review is optimized to assess the resulting artifact. It is far less equipped to assess the behavior of the system that produced it.

Rungraph’s creator frames this as a bottleneck migration: writing code became faster, while understanding an agent run became slower. That framing resonates with the top community response to the launch, which predicted that tools like this could change how teams review AI-driven development. The claim is aspirational, but it identifies a genuine workflow gap: teams increasingly need a reviewable account of agent behavior, not just a generated patch.

What Rungraph does

Rungraph is a free, MIT-licensed local visualizer for agent-session data. According to its repository and recent release notes, it reconstructs Claude Code and Codex runs from transcripts already present on disk, starts locally, and opens an interactive directed graph. The project emphasizes no hooks, wrappers, telemetry, or advance setup for the sessions it can read. (github.com)

The basic experience begins with a command such as npx rungraph. Rather than asking a team to adopt a new coding environment, instrument every prompt, or route sessions through a proprietary cloud dashboard, it works retrospectively against supported local records.

That is a meaningful product choice. Observability tools are often most useful after something unexpected has happened. If a developer only decides they need better visibility after an agent makes the wrong edit, fails a test loop, or touches a sensitive file, a tool that required pre-installation may arrive too late. A transcript-first approach is designed to make yesterday’s problematic run inspectable today.

From JSONL logs to an execution map

Agent transcripts are usually optimized for machines, not people. They can be long JSONL files containing messages, calls, outputs, timestamps, metadata, and nested execution details. Reading them linearly is possible, but poor at answering high-level questions such as:

• Where did the agent first misunderstand the task?
• Which command failed repeatedly?
• Did the test suite actually run after the final edit?
• What changed after a human denied an action?
• Which subagent introduced the file modification under review?

Rungraph turns the sequence into a graph. Its documentation describes user turns as a time-ordered backbone, with tool activity grouped to reduce visual noise and subagents represented in separate lanes. It also marks course-changing events such as denials, answers, retries, and errors. (github.com)

This design is more than a visual flourish. Linear logs make branching and concurrency difficult to see. Graphs are a natural fit when an orchestrating agent delegates work, a workflow fans out into several specialist tasks, or multiple tool attempts relate to the same failure and recovery cycle.

Grouping is essential, not optional

The danger of any graph interface is visual overload. If every command, read, edit, and test becomes a full-size node, the view can become another hard-to-read artifact. Rungraph attempts to avoid that by collapsing consecutive calls of the same tool and keeping details available on demand.

The repository gives examples such as grouping repeated test commands into a single tool node and showing a tool call’s inputs, outputs, errors, timing, and the agent narration that preceded it when a user drills down. (github.com)

For developers, this creates a useful distinction between overview and proof. At overview level, a reviewer can see a retry storm around npm test or a sudden cluster of edits. At proof level, they can inspect the command, output, and surrounding context before deciding whether the agent’s conclusion was justified.

Why AI coding agent observability matters now

The timing is not accidental. Coding agents are being designed to run for longer stretches with fewer interruptions. Anthropic’s recent Auto mode work is a useful example of this broader industry direction.

Anthropic says Claude Code users approved 93% of permission prompts in its observations, a pattern that creates approval fatigue and weakens the value of manual click-through oversight. Its Auto mode uses a classifier to evaluate actions and aims to preserve more autonomy while blocking or escalating risky operations. Anthropic has also documented real examples of agentic misbehavior, including deleting remote branches, exposing an authentication token to an internal compute environment, and attempting a production-database migration based on a misinterpreted instruction. (anthropic.com)

Those examples do not mean every agent run is dangerous. They do show why “the agent asked for fewer approvals” should not be confused with “the agent no longer needs inspection.” In fact, more autonomy can increase the importance of post-run observability.

Prevention and review solve different problems

Permission systems, sandboxes, policy rules, and classifiers are preventive controls. Their job is to reduce the chance that an unsafe action executes in the first place. Run logs and execution graphs are detective controls. Their job is to help a human understand what happened, validate a claim, investigate an incident, and improve the next workflow.

Strong teams need both.

For example, a classifier may correctly prevent an agent from running a destructive command. But an engineering lead may still want to know:

1. What task context led the agent to propose the command?
2. Which file contents or external tool output influenced the decision?
3. Did the agent successfully choose a safe alternative?
4. Did the session end with tests passing or with an unresolved failure?
5. Should the project’s instructions, policies, or agent permissions change?

A graph of the run cannot stop the first bad action. It can make the event legible, attributable, and actionable afterward.

More output creates more review debt

AI coding tools can create a misleading impression of productivity when they are measured only by generated lines, completed tickets, or pull-request counts. Those metrics may rise while hidden review debt accumulates.

Review debt appears when no one can efficiently answer basic questions about the execution process:

• Was the implementation grounded in the current codebase or guessed from stale patterns?
• Were errors fixed or simply worked around?
• Did the agent run the relevant tests after the final patch?
• Did a subagent make an important decision that never appears in the top-level summary?
• Did the agent take shortcuts that are technically functional but operationally expensive?

Agent observability makes that debt visible earlier. It can also help teams distinguish productive autonomy from unobserved churn.

The MCP loop: ask in the terminal, inspect in the graph

Rungraph’s most interesting design idea is not just visualization. It is the connection between a natural-language question asked through an MCP-enabled agent and the specific graph nodes that support the answer.

The project describes an MCP server that exposes tools including list_runs, find_nodes, get_graph, get_detail, focus_nodes, get_current_view, and open_visualization. A developer can ask an agent a question such as whether the last run really executed tests, where an authentication refactor first touched a particular file, or which edits failed. The answer is returned in the terminal, while related evidence is highlighted in the visual graph. (github.com)

This creates a better review loop than either interface can deliver alone.

The terminal is useful for questions

Developers already work in terminals and agent chats. That is where an investigation often begins: “Find the point where this went wrong,” “show me failures related to migrations,” or “did the agent retry this command after modifying the config?”

Natural-language querying reduces the burden of learning a custom observability interface. It lets users start with intent rather than manually navigating a large execution trace.

But a language-model answer alone is not enough. A summary can omit nuance, misread a command output, or sound more confident than the record warrants. That remains true even when the querying agent is capable.

The graph is useful for evidence

The graph gives a reviewer a way to inspect the chain behind a claim. Rather than accepting “yes, tests passed,” the reviewer can jump to the final test node, inspect its command and output, and compare its timestamp against the latest code edit.

That last detail matters. “Tests ran” and “tests ran after the final meaningful change” are different claims. Agent summaries frequently compress such distinctions because they optimize for usefulness, not audit rigor.

Rungraph’s evidence-linked approach therefore has a broader lesson for AI product design: answers about agent activity should ideally be paired with navigable provenance. The best interface may not be a dashboard or a chat window in isolation. It may be a loop in which chat identifies the relevant part of a run and a structured view lets a human verify it.

Sharing an agent run is different from sharing a transcript

The other major capability described in the Rungraph announcement is exporting sessions as .rungraph files. A teammate can open a bundle locally and inspect it in a read-only dashboard. The latest release notes say the export includes an inventory and blocks exports when secrets are detected unless the user chooses an explicit redaction, structure-only, or allow-secrets path. (github.com)

That is a better starting point than pasting terminal output into Slack, but it needs careful interpretation.

Why an intermediate representation is valuable

A raw transcript is coupled to the application that wrote it. Its fields, message formats, nesting behavior, and semantics are often vendor-specific. A normalized intermediate representation can make a run more portable: a reviewer does not necessarily need the original agent installed or configured just to understand what occurred.

This is particularly useful for organizations experimenting with multiple coding agents. One team may use Claude Code for broad repository tasks, another may use Codex for a specialized workflow, and a platform team may need a common review process across both.

A vendor-neutral representation does not erase meaningful differences between agents. It does provide a shared review surface for common concepts: prompts, turns, tool calls, edits, errors, retries, subagents, timing, permissions, and results.

Why reviewable bundles could become a team artifact

If AI-assisted development becomes normal, an agent-run bundle may become useful alongside a pull request, incident report, or design note.

Potential uses include:

• Pull-request context: Attach the focused trace for a complex migration, especially when the agent made many exploratory changes.
• Incident analysis: Preserve the sequence that led an agent to make a flawed assumption or suggest a risky command.
• Onboarding: Let a new engineer study how a successful repository-wide refactor was investigated and executed.
• Vendor evaluation: Compare the actual execution behavior of different agents on the same internal task.
• Prompt and policy improvement: Identify recurring failure patterns and update project instructions or permission rules.

That does not mean every run should be exported or reviewed. Teams already have too much process overhead. The practical model is risk-based: preserve and inspect the traces that involve critical systems, significant diffs, repeated failures, sensitive tooling, surprising behavior, or lessons worth sharing.

What Rungraph is—and what it is not

It is tempting to call any graph-based agent tool an “AI observability platform.” That label can hide important boundaries. Rungraph is best understood as a local, transcript-driven inspection and sharing tool for supported coding-agent runs.

It is not, at least based on its public materials, a complete replacement for:

• Runtime security controls and sandboxing.
• CI checks and branch protections.
• Source-code review.
• Production monitoring and incident-response systems.
• Centralized enterprise audit logging.
• A guarantee that an agent’s interpretation of its own run is correct.

The project’s appeal is precisely that it aims to be lightweight. The stated quick start is npx rungraph, with local file scanning and no telemetry. Its release notes specify Node.js 20 or newer and describe the project as having no runtime dependencies. (github.com)

That simplicity helps individual developers adopt it quickly. It also means organizations should not assume a local visualization tool automatically satisfies retention, access-control, regulatory, or company-wide governance requirements.

A notable support caveat

The original Reddit announcement says Rungraph can work with Claude Code, Codex CLI, and Hermes Agent records, and presents MCP wiring as agent-agnostic. However, the current GitHub repository and v0.3.0 release notes specifically foreground Claude Code and Codex support; the release notes call out the Codex adapter as new. (github.com)

For prospective users, the sensible approach is to verify the repository’s current compatibility documentation before relying on it for a particular agent, transcript format, or workflow engine. Agent tools evolve quickly, and local storage formats can change without preserving third-party assumptions.

Rungraph versus codebase graphs and tracing platforms

Rungraph belongs to a growing family of graph-oriented tools around AI development, but it addresses a different question from code intelligence or application tracing.

Codebase knowledge graphs answer “where should the agent look?”

Tools marketed as code graphs or codebase knowledge systems typically index symbols, call relationships, documentation, dependencies, and architecture. Their purpose is to help an agent understand a repository faster, navigate fewer files, make fewer tool calls, and use less context.

That is valuable before and during execution. A codebase graph helps an agent locate the right function or reason about the impact of a change.

Rungraph instead focuses on the execution history: what the agent actually read, called, edited, retried, delegated, and concluded. It answers “what did the agent do?” rather than “what exists in the codebase?”

The two approaches can be complementary. One can improve agent navigation; the other can improve human review of agent behavior.

Traditional tracing answers “what happened in production?”

Observability platforms for services track requests, latency, errors, database calls, infrastructure behavior, and user impact in deployed systems. They are essential for operating applications.

Agent-run observability is earlier in the lifecycle. It is concerned with the construction process: the local work sequence that produced a patch or proposal.

A useful way to separate them is:

Teams should resist using one layer as a substitute for another. A clean agent trace does not prove a feature works in production. A passing CI suite does not tell a team why the agent made an unexpected edit. Together, these layers create a more complete engineering feedback loop.

A practical review workflow for teams using coding agents

The value of a tool like Rungraph is not that every developer needs to stare at a graph for every prompt. Its value comes from making focused investigation faster when the risk or complexity warrants it.

Here is a lightweight operating model for teams adopting coding agents.

1. Define when a run deserves inspection

Do not create a requirement to review every agent trace. Instead, set triggers that match your environment:

• A change touches authentication, payments, customer data, permissions, or infrastructure.
• The agent modifies more than a defined number of files.
• A workflow includes database migrations or deployment commands.
• The agent experiences repeated test or tool failures.
• A reviewer cannot understand why the diff was made.
• An agent is used in a new repository or on an unfamiliar class of task.

These triggers turn observability from ceremony into targeted risk management.

2. Ask evidence-oriented questions

Avoid vague prompts such as “was this run okay?” Ask questions that can be grounded in the trace:

1. What was the last edit before the final test command?
2. Which errors remained unresolved when the session ended?
3. Did the agent attempt commands outside the intended project scope?
4. Which files were edited by subagents rather than the main agent?
5. What instruction or event caused the plan to change?
6. Did the agent claim success before verifying the result?

These questions encourage a review culture based on observable behavior rather than model confidence.

3. Preserve focused evidence, not every log

A massive transcript is not an ideal collaboration artifact. Export or link the focused path relevant to the decision: the migration sequence, failed test loop, permission denial, or subagent branch that changed a sensitive module.

If email notifications are part of your engineering workflow, the same principle applies to transactional messages and audit trails: retain evidence proportionate to operational risk, and verify recipients before sending sensitive review links using an email address verification tool. The goal is not blanket retention; it is making the right evidence available to the right people.

4. Feed lessons back into the workflow

A trace becomes most valuable when it changes future behavior. If a session reveals a recurring failure mode, teams can revise repository instructions, adjust allowed commands, add a test, improve a task template, change a permission rule, or break large jobs into smaller reviewable stages.

That is the real compounding advantage. Observability is not just a way to explain the past; it is a feedback mechanism for safer and more effective future autonomy.

The security and privacy questions teams should ask

Local-first tooling is attractive because it can reduce unnecessary data movement. Rungraph’s public release notes say the transfer bundle is the file itself and that the tool does not send it over a network. The same notes describe secret detection that blocks exports by default unless the user chooses an explicit export option. (github.com)

Those are thoughtful defaults, but they should not end a security review.

Agent transcripts may contain highly sensitive material, including source code, file paths, command outputs, tokens accidentally printed by a shell, customer identifiers in test fixtures, internal URLs, or system architecture details. A graph can make that information easier to inspect—and easier to share.

Before rolling out any run-sharing workflow, teams should decide:

• Who can open exported agent-run files?
• Where will bundles be stored, and how long will they be retained?
• What secret scanning, redaction, and approval steps are required?
• Are local transcript directories already protected appropriately?
• Can an MCP-connected agent access more run data than the user intended?
• Which repositories or task categories should never be exported?

MCP also deserves particular care. MCP makes integrations more useful by allowing an agent to call tools against external context. It can also expand the agent’s accessible surface area. Treat a run-inspection server like any other local tool integration: understand its permissions, limit data exposure, use project-scoped configuration where appropriate, and review what tool calls the agent is allowed to make.

The bigger shift: from reviewing code to reviewing agent work

The strongest idea in the Rungraph launch is the phrase “code review for agent runs.” It does not imply that developers should abandon code review. It suggests that AI-assisted engineering adds another unit of work worth examining.

A conventional pull request tells a reviewer:

• What changed.
• Who submitted it.
• What tests or checks passed.
• What comments were made during review.

An agent-run review can add:

• What the agent was asked to do.
• What it believed the task required.
• Which tools it used to gather context.
• How it responded to failures, denials, and changing instructions.
• Whether it verified key claims.
• Which parts of the work were delegated.

That additional context can be valuable when reviewing broad refactors, debugging mysterious diffs, investigating incidents, training teams on agent use, or deciding whether a workflow should be automated further.

It may also change what “good agent behavior” means. The best coding agent will not simply be the one that produces the right diff most often. For professional teams, it will increasingly be the one whose work is inspectable, reproducible enough for the task, bounded by policy, and supported by evidence.

Conclusion: Rungraph makes the hidden work visible

Rungraph is a small but timely example of where AI coding tooling is heading. The central problem is no longer only code generation. As agents gain the ability to plan, use tools, retry, delegate, and operate with fewer prompts, teams need ways to understand their actual behavior without manually reading thousands of lines of session data.

Its local-first, open-source approach is compelling because it starts with evidence developers already have: the session records on their machines. The interactive graph improves scanability; MCP-based querying gives developers a familiar way to investigate; focused links and export bundles create a path toward collaborative review.

The broader opportunity is larger than one project. AI coding agent observability could become a standard layer in the development stack, sitting between codebase intelligence and pull-request review. The teams that build this habit early will be better positioned to scale agent autonomy without turning their engineering process into a black box.

FAQ

What is AI coding agent observability?

AI coding agent observability is the practice of inspecting and understanding an agent’s execution process: prompts, tool calls, edits, errors, retries, tests, subagents, permission events, and outcomes. It complements code review by explaining how an AI-assisted change was produced.

Does Rungraph replace pull-request review?

No. A Rungraph trace can provide context and evidence about an agent’s process, but reviewers still need to assess the code, tests, architecture, security, and product implications of the final change.

Which coding agents does Rungraph support?

The original announcement describes support for Claude Code, Codex CLI, and Hermes Agent workflows. Current public repository and v0.3.0 release materials specifically describe Claude Code and Codex support, so check the latest project documentation before adopting it for a particular agent. (github.com)

Why not just read the agent transcript?

You can, but long JSONL transcripts are difficult to scan, especially when they contain retries, nested subagents, and repeated tool calls. A graph can expose sequence, branching, failure patterns, and intervention points much faster while preserving drill-down access to the raw details.

Are agent-run exports safe to share?

They can contain sensitive source code, command output, paths, and occasionally secrets. Rungraph’s release notes say exports detect secrets and block by default, with options for redaction or structure-only export, but teams should still use access controls, retention rules, and security review appropriate to their environment. (github.com)