MCP Code Mode: Why AI Agents Need Fast Tool Batching With Checkpoints

MCP Code Mode is one of the most important ideas in the current AI-agent tooling debate because it challenges a costly assumption: that an LLM must stop, reason, emit a tool call, read the response, and repeat for every API interaction. Cloudflare’s approach is faster and more context-efficient—but, as researcher Yannic Kilcher argues in a response to Theo’s critique of MCP, speed alone does not solve the problem of unpredictable real-world workflows. (youtube.com)

The real MCP Code Mode debate is about execution, not protocols

The debate can sound like a referendum on the Model Context Protocol itself. It is not. MCP is a standardized way for AI applications to connect with external tools, data, prompts, and other capabilities through a client-host-server architecture. It gives agents a common interface for reaching systems such as CRMs, databases, deployment platforms, calendars, internal knowledge bases, and analytics tools. (modelcontextprotocol.io)

That standardization matters. Without it, every AI client, SaaS provider, and developer framework needs a custom integration layer. But MCP does not inherently make an AI model better at reasoning, calling APIs correctly, validating data, or recovering from failures. It is plumbing—not intelligence.

That distinction is central to the response video from Yannic Kilcher, titled What Cloudflare’s code mode misses about MCP and tool calling. The video responds both to Cloudflare’s original Code Mode article and to Theo “t3.gg” Browne’s video, MCP is the wrong abstraction. Kilcher broadly agrees with the code-first direction while challenging a stronger conclusion: that agents can safely avoid intermediate model reasoning whenever they need to chain tools together. (youtube.com)

The useful takeaway for builders is more nuanced than “MCP is wrong” or “tool calling is obsolete.” The key question is:

Which parts of an agent workflow are predictable enough to compile into code, and which parts require a model to inspect reality before choosing the next move?

That question affects latency, token usage, reliability, observability, permissions, and ultimately whether an AI workflow is safe enough to automate.

What Cloudflare means by MCP Code Mode

Traditional tool calling exposes a list of tool definitions to a model. Each definition typically includes a name, a natural-language description, and an input schema. The model selects a tool, produces structured arguments, receives a result, and then decides what to do next.

Cloudflare’s MCP Code Mode changes the interface. Instead of asking the model to interact directly with a large catalog of tool schemas, it converts MCP tools into a typed TypeScript API. The model writes code against that API, and the runtime executes the code. (blog.cloudflare.com)

At a high level, the difference looks like this:

1. Direct tool calling: The model decides to call getCustomer, waits for the result, reads it in context, then calls getInvoices, waits again, and finally produces an answer.
2. Code Mode: The model writes a short TypeScript program that fetches the customer, gets relevant invoices, filters them, calculates a total, and returns only the needed result.

The second approach treats the model less like a chatbot issuing isolated commands and more like a developer writing a small integration script.

Why TypeScript can be a better interface for LLMs

Cloudflare’s hypothesis is intuitive: models have seen enormous amounts of code during pretraining, including API clients, SDK usage, type definitions, loops, error handling, data transformations, and common library patterns. By comparison, the exact tool-calling formats used by modern LLM platforms are relatively new and may rely more heavily on post-training behavior. (blog.cloudflare.com)

This does not mean models are inherently perfect TypeScript programmers. They are not. But it does mean that typed functions, composable calls, variable names, and familiar control flow often provide a more natural “language” for an LLM than dozens or hundreds of JSON tool schemas.

Anthropic has independently described a closely related pattern: allow agents to use code execution with MCP so they can discover tools on demand, make calls from code, and avoid forcing every intermediate result through the model’s context window. Anthropic says this can reduce context overhead substantially in tool-heavy workloads. (anthropic.com)

The context-window problem Code Mode addresses

The immediate advantage is not philosophical. It is economic and operational.

A large tool catalog can consume a significant share of an agent’s context before the user has even asked a question. Tool descriptions, parameter schemas, examples, and intermediate outputs all compete with task instructions, retrieved documents, conversation history, and the model’s own working space.

Cloudflare later demonstrated the issue with its own API surface: it says the Cloudflare API has more than 2,500 endpoints, and exposing every endpoint as an MCP tool could require more than two million tokens. Its newer server-side Code Mode design instead exposes two top-level tools—search and execute—and keeps the initial context footprint around 1,000 tokens. Cloudflare estimates a 99.9% reduction in input-token usage for that specific comparison. (blog.cloudflare.com)

The exact savings will vary by API design and task, but the broader lesson is durable: tool metadata is context, and context is a budget.

Why batch execution is genuinely powerful

Cloudflare’s strongest argument is about multi-step work. In a normal agent loop, each step may require another model inference:

• Call a tool.
• Insert the result into context.
• Ask the model what it means.
• Generate another tool call.
• Repeat.

That can be unnecessarily expensive when the relationship between steps is straightforward. If an agent needs to retrieve a user ID, fetch that user’s active subscriptions, sum the charges, and format a short answer, a single generated script can handle the sequence without repeatedly asking the LLM to copy data from one call into the next.

A simple example: deterministic API composition

Imagine a support agent answering: “What is Acme Corp’s outstanding balance?”

A code-first plan could look conceptually like this:

const account = await crm.findAccount({ name: "Acme Corp" });
const invoices = await billing.listInvoices({ accountId: account.id, status: "open" });
const balance = invoices.reduce((total, invoice) => total + invoice.amountDue, 0);
return { account: account.name, balance, invoiceCount: invoices.length };

This is a highly suitable Code Mode task when the following are true:

• findAccount reliably returns one account or a well-defined empty result.
• The returned account ID is the correct input for billing.
• The invoice API has stable semantics.
• The final calculation is deterministic.
• The operation is read-only or otherwise safely authorized.

In this category of workflow, forcing the LLM to inspect every response adds overhead without necessarily adding judgment. The generated program is a compact execution plan, and the agent needs to see only a concise final result.

The practical gains for builders and marketers

For founders, creators, and marketing teams using agents across many SaaS systems, this matters more than it may first appear. A typical AI automation may touch analytics, ad platforms, ecommerce, email, CRM, spreadsheets, content libraries, and project-management tools.

Code Mode can make several useful workflows cheaper and faster:

• Campaign reporting: Pull spend from ad platforms, revenue from ecommerce, and conversion data from analytics, then calculate channel-level ROAS.
• Content operations: Query a content database, identify pages with declining traffic, collect ranking keywords, and draft a prioritized refresh brief.
• Customer research: Combine CRM segments, support-ticket tags, call summaries, and product-usage data into a concise account review.
• Developer operations: Check deployment status, read error rates, inspect a feature flag, and create a structured incident summary.
• Finance administration: Reconcile invoices, subscriptions, and payment records when the schemas and business rules are explicit.

The important qualifier is that these are not just “multi-step” tasks. They are multi-step tasks with predictable transitions.

The weakness in fully planned agent workflows

Kilcher’s critique begins where the clean API demo ends. In the real world, intermediate results often do not simply fill variables. They change the meaning of the task.

A user-location example makes the point. In a neat demo, an agent can call getLocation(), pass the result into getWeather(location), then recommend clothing. But real location data may be missing, stale, ambiguous, contradictory, privacy-restricted, or expressed in a form the weather service cannot use.

The first tool may return any of the following:

• A precise latitude and longitude.
• A city inferred from a previous session.
• An address without a region or country.
• A statement that no current location is available.
• A conflict between device location and calendar location.
• A request for user clarification.

At that point, the next best action is not merely a type conversion. It requires interpretation. Should the agent ask the user? Use the last known location? Check a calendar? Decline to infer? Those are policy and reasoning decisions.

Data can be schema-valid and still be wrong

This is the blind spot in many agent-system diagrams. JSON schemas improve syntactic reliability, but they cannot guarantee semantic correctness.

A field called customer_id may be present and valid while referring to a duplicate record. A search endpoint may return ten plausible matches. An API may return a successful HTTP status with stale data. An inventory service may represent “not available” as zero, null, omitted, or a delayed fulfillment date. A support system may classify an urgent security issue as a generic ticket because an upstream tagger failed.

All of those outputs can be machine-readable. None are automatically safe to feed into the next action.

Kilcher’s core objection is therefore not that preplanned code is bad. It is that a fully deterministic plan silently assumes the agent will never encounter an intermediate state that should change the plan. In complex workflows, that assumption will eventually fail. (youtube.com)

Planning is not the same as adapting

This is a familiar engineering trade-off. A batch job is efficient because it minimizes coordination. An interactive controller is resilient because it observes the system and adapts as conditions change.

Agent builders should treat this as a spectrum:

• At one end are fixed pipelines, where a predetermined sequence is ideal.
• At the other are open-ended investigations, negotiations, incident responses, and research tasks, where each observation can redirect the work.
• Most valuable production workflows sit in the middle.

The mistake is framing the choice as code versus tool calling. The real choice is how much uncertainty the system can safely defer until after execution.

Speculative tool calling: the hybrid worth testing

Kilcher proposes a useful middle path that can be described as speculative tool calling. The agent writes and executes a batch of likely tool calls, but the runtime records intermediate inputs, outputs, and decisions. The model can then review that trace, identify suspicious deviations, and decide whether the final result is trustworthy or whether a branch needs to be re-run. (youtube.com)

The idea resembles speculative execution in systems design: do useful work ahead of time, then validate whether the assumptions behind it held.

How speculative tool calling could work

A practical implementation might use five stages:

1. Generate a bounded plan. The model writes code for a sequence of low-risk, likely operations.
2. Execute in a sandbox. The runtime applies strict timeouts, permissions, network boundaries, and resource limits.
3. Capture a structured trace. Store each call, its arguments, response metadata, status, and a truncated or summarized result.
4. Run deterministic validators first. Check schemas, IDs, freshness, policy rules, confidence thresholds, anomalies, and side-effect eligibility.
5. Escalate to model review only when needed. Give the LLM a compact trace and ask whether a result is ambiguous, inconsistent, or unsafe to continue from.

This avoids a full model round trip after every predictable operation while retaining a feedback loop when the environment becomes messy.

A campaign-analysis example

Suppose a marketing agent receives this request: “Explain why paid search conversions fell last week and suggest what to do.”

A naïve batch plan might:

1. Retrieve Google Ads spend and conversions.
2. Retrieve analytics sessions and conversion rate.
3. Retrieve landing-page performance.
4. Retrieve recent campaign-change history.
5. Summarize the difference week over week.

This is a reasonable speculative batch. But imagine the intermediate data shows that analytics tracking was disabled for two days, one campaign was paused by a human, and a landing page returned an unusually high number of 404s after a deployment.

A reliable system should not blindly produce “increase bids.” It should recognize that the observed conversion decline may be measurement-related, operational, or caused by site errors. The next actions could become:

• Verify tagging status and data completeness.
• Check deployment logs and page availability.
• Separate paused campaigns from active campaigns.
• Ask whether the user wants recommendations for recovery versus root-cause analysis.

That is exactly where an intermediate checkpoint earns its cost.

Build checkpoints around uncertainty, not arbitrary step counts

A common implementation mistake would be to validate every three tool calls, every five API requests, or whenever a script finishes. Those rules are easy to implement but poorly aligned with actual risk.

The better approach is to define checkpoints around uncertainty boundaries.

Good checkpoint triggers

Pause or route a workflow to validation when any of these conditions occur:

• A search returns multiple plausible entities.
• A required field is missing, null, stale, or outside an expected range.
• Two systems disagree about the same fact.
• A tool output changes the objective, not just an input value.
• An operation would create, delete, publish, spend, deploy, or message someone.
• The workflow crosses a permissions boundary or accesses sensitive data.
• A request is partial, ambiguous, or conflicts with stored user preferences.
• A model-generated query would affect more records than an approved threshold.
• A downstream call depends on a low-confidence inference.

The goal is not to eliminate uncertainty. It is to make uncertainty explicit, measurable, and governable.

A useful three-lane architecture

For many production agents, a three-lane model is more practical than a single universal execution strategy:

A content-marketing agent can use the deterministic lane to assemble traffic data, the conditional lane to assess whether keyword cannibalization is real, and the high-impact lane before publishing changes to production pages.

This pattern is much more useful than declaring either MCP tools or agent-written code universally superior.

Code Mode does not eliminate MCP—it changes what MCP exposes

The claim that MCP is “the wrong abstraction” is provocative, but it can obscure the architectural layering at work.

MCP defines a protocol for exposing capabilities. It does not mandate that every underlying API endpoint be presented directly to the model as a separate tool. The MCP specification explicitly leaves room for different interface patterns, even though tools are designed to be model-controlled. (modelcontextprotocol.io)

Code Mode can therefore be understood as an MCP design pattern rather than an MCP replacement.

Instead of exposing 2,500 raw API operations, an MCP server might expose:

• A discovery function for finding the relevant parts of an API or SDK.
• A constrained execution function for running generated code.
• A small number of higher-level domain actions.
• Read-only reporting functions for common use cases.
• Approval-aware operations for sensitive writes.

Cloudflare’s newer implementation makes this architecture concrete by reducing a huge API surface to search and execute tools backed by Code Mode. That is not abandoning MCP. It is using MCP as a narrow control plane while moving detailed API composition into a typed, sandboxed execution layer. (blog.cloudflare.com)

For product teams, this suggests an important design principle: do not equate the number of MCP tools with the richness of your product’s agent integration. A smaller, well-designed interface can be more capable in practice if it lets the model discover operations progressively and operate within clear guardrails.

Security becomes more important when agents write code

MCP Code Mode improves expressiveness, but it also raises the stakes. Allowing an LLM to generate and run code means the system needs strong containment—not merely a clever prompt.

The official MCP specification emphasizes that the protocol can enable arbitrary data access and code-execution paths, and it calls for explicit user consent, clear control over operations, and careful privacy protections. (modelcontextprotocol.io)

Cloudflare likewise notes that agent-generated code needs a secure place to run. Its work on sandboxing agents highlights the operational requirement behind the Code Mode vision: code execution is only useful if it is isolated, constrained, observable, and fast enough for real workflows. (blog.cloudflare.com)

Minimum controls for a Code Mode deployment

If you are building this pattern, treat the following as baseline requirements rather than optional enhancements:

• Least-privilege credentials: Issue short-lived, scoped tokens. A reporting workflow should not have deployment or billing permissions.
• Sandbox isolation: Prevent arbitrary filesystem access, unrestricted outbound network access, secret exfiltration, and process escape.
• Allowlisted packages and APIs: Do not allow model-written code to import anything available on the public internet by default.
• Read/write separation: Make read operations easy; make state-changing operations explicit and harder to invoke.
• Budget limits: Cap runtime, memory, number of API calls, data volume, retries, and spend.
• Audit logs: Record generated code, tool calls, arguments, policy decisions, user approvals, and final outputs.
• Human confirmation for material actions: Require approval before sending customer messages, changing ad budgets, deleting data, altering DNS, or deploying production code.
• Data minimization: Return only the fields and summaries necessary for the task, rather than forwarding full raw records into model context.

These controls are not anti-agent friction. They are what make agents deployable outside a demo environment.

The community reaction: agreement on the problem, disagreement on the boundary

There were no top-comment reactions supplied with the original source, so the clearest visible discussion comes from the three linked positions: Cloudflare’s product argument, Theo’s critique of MCP’s current tool abstraction, and Kilcher’s caution about deterministic execution. (youtube.com)

Despite their different framing, there is substantial agreement:

• Loading massive tool catalogs into context is inefficient.
• Models often work more naturally with code-like interfaces than with giant lists of isolated JSON functions.
• Passing every intermediate result through the model is costly.
• Developers need better ways to discover tools dynamically and compress context.
• MCP itself should not be treated as a magical source of new agent capabilities.

The disagreement is about the boundary of automation. Cloudflare’s Code Mode argument emphasizes that generated code can efficiently compose API calls and return only the answer the model needs. Kilcher’s response emphasizes that agents often need to see intermediate outputs because the world is not a clean typed pipeline.

Anthropic’s related work points toward a practical synthesis rather than a winner-take-all answer. It advocates tool discovery on demand and code execution for efficiency, while its broader agent guidance stresses context management, well-designed tools, and robust harnesses for multi-step work. (anthropic.com)

In other words, the emerging expert consensus is not “always use code” or “always keep the model in the loop.” It is: use code for compression and execution, then use model judgment and deterministic checks where the evidence demands it.

How to decide whether MCP Code Mode fits your workflow

Before adopting Code Mode, map a workflow according to its structure rather than its buzzword appeal.

Code Mode is a strong fit when

• The API surface is large but coherent.
• Operations can be discovered from typed SDKs or OpenAPI-style descriptions.
• The workflow contains repetitive reads, joins, filters, pagination, aggregation, and formatting.
• Intermediate outputs are mostly machine-consumable inputs to the next step.
• You can define clear error handling and fallback behavior.
• The environment supports secure sandboxing and granular credentials.
• The task benefits from returning a concise result rather than raw records.

Keep interactive tool calling or explicit checkpoints when

• Search results are ambiguous or require subjective selection.
• The next action depends on qualitative interpretation.
• Data sources are incomplete, stale, contradictory, or loosely structured.
• The agent must negotiate requirements with a user.
• The workflow involves legal, financial, health, security, or reputational consequences.
• Side effects are expensive or hard to reverse.
• A wrong action could create a silent failure that looks successful in logs.

For most teams, the correct answer will be hybrid. Start with a small batchable core, instrument failures, identify the moments where plans diverge, and add checkpoints there.

A practical rollout plan for founders and builders

The best way to test MCP Code Mode is not to replace your existing agent loop all at once. Treat it as an optimization and reliability project.

Phase 1: Measure the current workflow

Instrument an existing tool-using agent and capture:

• Tokens spent on tool definitions.
• Tokens spent on intermediate tool results.
• Number of model round trips per task.
• End-to-end latency.
• Tool-call success and retry rates.
• Human correction rate.
• Frequency of ambiguous or malformed outputs.
• Cost per completed task.

You cannot make a credible Code Mode business case without knowing whether context overhead or tool orchestration is actually your bottleneck.

Phase 2: Choose one read-heavy workflow

Pick a workflow with limited downside, stable APIs, and a clear success metric. Good candidates include weekly reporting, account summaries, content inventory analysis, or engineering-status rollups.

Avoid starting with payment actions, customer communications, infrastructure changes, or autonomous publishing. Those may eventually benefit from the same architecture, but they are poor first experiments.

Phase 3: Build a typed execution layer

Create a small SDK-like interface with:

• Clear function names.
• Strong input and output types.
• Useful errors.
• Pagination helpers.
• Safe defaults.
• Built-in rate-limit handling.
• Redacted logging.

Do not simply wrap every REST endpoint one-to-one. Model-facing APIs should reflect useful domain tasks and safe compositional primitives.

Phase 4: Add trace review and fallbacks

Log the generated program and each intermediate result. Define automatic stop conditions for empty results, ambiguous matches, stale data, anomalous values, permission errors, and unexpectedly broad actions.

Then compare three variants in evaluation:

1. Traditional sequential tool calls.
2. Unchecked batch Code Mode.
3. Batch Code Mode with deterministic validators and model checkpoints.

The third option may not be the cheapest in every case, but it is the architecture most likely to reveal where speed and reliability genuinely trade off.

The bottom line: compile certainty, inspect uncertainty

MCP Code Mode is a valuable correction to the simplistic agent loop. Giving an LLM a typed API and letting it write short programs can reduce tool-schema bloat, eliminate unnecessary model round trips, and unlock more capable use of large API surfaces. Cloudflare’s later implementation for its thousands of API endpoints shows why the pattern is gaining momentum. (blog.cloudflare.com)

But Kilcher’s response identifies the essential limit: a code plan created before execution cannot anticipate every meaningful state revealed during execution. Real data is messy, APIs are imperfect, and the right next action often depends on what just happened. (youtube.com)

The most useful design rule is simple: compile certainty, inspect uncertainty. Batch deterministic work. Validate intermediate states that could alter the plan. Keep a model—and, for consequential actions, a human—at the decision points where interpretation matters.

That is not a compromise born from indecision. It is how mature agent systems will deliver Code Mode’s speed without confusing a fast execution trace for a reliable outcome.

FAQ

What is MCP Code Mode?

MCP Code Mode is an agent-integration pattern in which MCP-exposed capabilities are represented as a typed API, often in TypeScript, and an LLM writes code to call that API instead of issuing one conversational tool call at a time. (blog.cloudflare.com)

Is MCP Code Mode better than normal tool calling?

It is better for many deterministic, multi-step workflows because it can reduce repeated model calls and context usage. It is not automatically better for tasks where each intermediate result requires interpretation, clarification, or a change in strategy.

Does Code Mode replace the Model Context Protocol?

No. MCP is a protocol for connecting AI applications to tools and data. Code Mode can sit on top of MCP by changing how a server exposes and executes its capabilities. (modelcontextprotocol.io)

What is speculative tool calling?

Speculative tool calling is the proposed hybrid approach of executing a likely sequence of tool calls in a batch, retaining intermediate traces, and then validating whether those intermediate results should have changed the plan. It aims to preserve much of Code Mode’s speed while catching flawed assumptions. (youtube.com)

What is the biggest risk of letting an AI agent write API code?

The largest risks are excessive permissions, unintended side effects, data leakage, and hidden errors caused by ambiguous or misleading intermediate outputs. Use sandboxing, scoped credentials, policy gates, audit logs, and human approval for meaningful actions. (modelcontextprotocol.io)