Deterministic AI Workflows: How to Cut LLM Costs Without Losing Capability

Deterministic AI workflows are becoming one of the clearest ways for AI product teams to lower inference costs without making their products feel less intelligent. A recent builder story from the 3D-generation space illustrates the pattern: keep the LLM responsible for interpreting intent and selecting actions, then hand repeatable, precision-heavy work to conventional software.

The lesson is bigger than 3D modelling. It applies to support agents, content pipelines, sales operations, document processing, developer tools, marketing automation, and nearly every SaaS product currently discovering that an impressive prototype can become an expensive production system.

The costly trap: asking an LLM to do the entire job

The original example comes from a post on r/SaaS by the team behind Graphyti, a product for generating 3D worlds from text prompts. According to the team, its first implementation asked a language model to generate all of the 3D code needed to create terrain, cities, and buildings.

It worked in the narrow sense that the product produced output. But a complex building reportedly took around 15 minutes, consumed roughly 4.5 million tokens, and could produce materially different results when run again with the same prompt. That combination is difficult for any SaaS company to live with for long.

High token consumption is an obvious concern, but it is not the only one. A workflow that creates a fresh pile of code on every request has several hidden operating costs:

• Latency becomes a product problem. Users do not care whether a result is slow because the model is reasoning, generating code, retrying, or waiting for downstream execution. They only experience a long wait.
• Output variation becomes a support problem. If the same request produces a different building, report, workflow, or email each time, customer success teams have a much harder time diagnosing complaints.
• Unbounded output becomes a margin problem. A product price may look profitable at average usage but become loss-making when a customer submits unusually complex prompts.
• Generated code becomes an engineering problem. Code needs validation, sandboxing, testing, execution, observability, error handling, and often repair loops.
• Model quality becomes a product dependency. A provider update, prompt change, or model substitution can alter the behavior of a workflow that was never truly specified in software.

The issue is not that LLMs are bad at code. They can be extremely useful for code generation, planning, transformation, and interpretation. The issue is architectural: using a probabilistic text generator as the runtime engine for a task that is fundamentally a known, repeatable computation.

What changed in the Graphyti architecture

Graphyti’s team inverted the responsibility split. Instead of requesting that the model construct the whole 3D asset from scratch, it now asks the model to make higher-level decisions: identify the building type, select a style, decide height and placement, and determine where roads should go. Those decisions are passed to purpose-built generators that perform the geometric work.

The team says that, for the same type of complex building, this lowered runtime to about seven minutes and token use to around 1.6 million tokens. That is approximately a 64% reduction in tokens and a 53% reduction in reported generation time, based on the figures shared in the post.

More importantly, the revised system returns the same output from the same inputs. In other words, the LLM may help translate a vague user request into a structured specification, but the actual world-building process is deterministic once that specification is accepted.

That is the essential model behind deterministic AI workflows:

1. A user provides unstructured intent in natural language.
2. The LLM interprets that intent and produces constrained structured data.
3. Validation checks whether the request is complete, safe, and within supported bounds.
4. Normal software, rules engines, APIs, database queries, or procedural generators execute the work.
5. The system records the inputs, version identifiers, and output so it can be rerun and audited.

This is not an anti-AI design. It is a more selective use of AI. The model does what it is uniquely good at—mapping human language to useful choices under ambiguity—while software does what it is uniquely good at—performing explicit operations reliably and cheaply.

Why deterministic AI workflows reduce costs so sharply

The immediate saving comes from fewer tokens. Rather than generating pages of code, long chains of intermediate reasoning, or repetitive implementation details, the model can return a compact plan such as a building class, a footprint, a height range, a façade style, a road connection, and a seed.

But the larger financial improvement comes from removing repeat work. Every time a model must restate conventions, write boilerplate, describe known transformations, or revise executable code after an error, the application is spending tokens to recreate logic that could have been stored in a function.

LLMs should choose parameters, not redraw machinery

Consider a request such as: “Create a dense coastal Mediterranean town with narrow streets, a harbor, and several four-story apartment blocks.” A fully generative system may attempt to write detailed geometry instructions, choose mesh primitives, position thousands of objects, create variations, and troubleshoot its own output.

A deterministic architecture can instead convert that request into a compact scene plan:

{
  "biome": "coastal_mediterranean",
  "settlement_density": "high",
  "street_pattern": "organic_narrow",
  "harbor": true,
  "building_types": ["apartment", "shopfront", "warehouse"],
  "apartment_height_range": [3, 4],
  "seed": 841029
}

The application’s generators can then create terrain, lay roads, allocate parcels, choose compatible assets, apply rules for roofs and façades, and render the result. The LLM is still essential because it interpreted the user’s language and resolved intent. It simply did not waste context on repeatedly expressing known procedural steps.

This division mirrors how mature software systems work elsewhere. A user may say “send an onboarding sequence to new trial users who did not activate within three days.” An LLM can parse the instruction, but an application should use known audience rules, event timestamps, templates, suppression lists, and delivery APIs—not have the model write a bespoke campaign engine on each request.

For teams building notification products, the same principle extends to every send: validate recipients before delivery, route a message through known logic, and make the model’s role optional rather than foundational. A free email address verification tool is a small example of a task where a deterministic check is usually more appropriate than asking a model to infer whether an address is deliverable.

Cost predictability matters more than average cost

Founders often track average cost per request, but variance is just as important. A user who enters a short, ordinary prompt may be inexpensive. A user who asks for an elaborate scene, uploads a huge document, requests multiple revisions, or triggers repair loops may consume vastly more resources.

That makes fixed pricing risky. If a product does not know the upper cost of a customer action, it cannot confidently decide where to put usage limits, which features belong in each plan, or whether a premium account is actually profitable.

Deterministic execution puts a ceiling on expensive work. A generator can have maximum map size, asset count, polygon budget, execution time, and retry count. A workflow can reject unsupported combinations before spending additional inference. The model may still be variable in its initial interpretation, but the operationally expensive part of the system has a defined envelope.

Determinism is also a debugging and trust feature

The Graphyti post emphasizes reproducibility: if a customer reports a problem, the team can reproduce the same output using the same inputs. This is an underrated benefit of hybrid AI systems.

In a purely prompt-driven product, “the same input” is often not enough to reproduce an issue. The result can also depend on the model version, temperature, system prompt, tool availability, retrieved context, hidden retries, API changes, timing, and the state of external services. If the LLM generated code, the exact code may also have changed.

A deterministic AI workflow does not eliminate all of those factors, but it makes them manageable. The application can store a durable execution record containing:

• the raw user request;
• the normalized structured plan;
• schema and prompt versions;
• model identifier and configuration;
• tool or generator version;
• input asset versions;
• random seed, where controlled variation is desired;
• validation outcomes;
• final artifact identifiers and logs.

With that record, support does not have to say, “We cannot recreate it because the model gave a different answer today.” Engineering can replay the plan against the relevant generator version, compare outputs, identify a faulty rule, and fix the problem.

Repeatability does not mean every result must look identical

There is an important distinction between determinism and sameness. Creative products often need variety. A city generator should not produce the exact same town for every customer, and a marketing assistant should not make every landing-page draft sound alike.

The solution is controlled randomness. Use a seed as an explicit input, not an accidental side effect of model sampling. The product can generate five distinct options by running the same system with five stored seeds. A customer can choose one, edit it, and later reproduce it precisely.

That design gives users creative variation while preserving supportability. It also makes A/B tests more meaningful because teams can vary one parameter or seed rather than unintentionally changing the entire underlying process.

The practical boundary: what should stay inside the LLM?

The hard part is not agreeing that functions are cheaper than long model outputs. The hard part is deciding where to draw the line.

A useful rule is this: use the LLM where the input is ambiguous, semantic, contextual, or open-ended; use deterministic software where the required operation is precise, repetitive, measurable, or governed by rules.

Good jobs for the model

LLMs add real value when a system needs to understand intent rather than execute a formula. Typical examples include:

• turning a natural-language request into a product configuration;
• classifying a customer’s issue before routing it;
• extracting meaning from messy documents;
• generating an initial creative brief or content angle;
• choosing among a finite set of tools or workflow branches;
• explaining output in language appropriate to the user;
• identifying missing information and asking clarifying questions;
• proposing exceptions when strict rules do not cover a case.

These are tasks where a rigid interface can make a product difficult to use. Natural language is the convenience layer that lets customers express goals without learning every field, filter, or setting.

Good jobs for deterministic systems

Move work out of the model when it can be described as a reliable operation with accepted inputs and outputs. That often includes:

• calculations, sorting, filtering, and aggregations;
• database reads and writes;
• API calls with documented parameters;
• pricing, discount, tax, eligibility, or compliance rules;
• rendering pipelines and asset placement;
• file conversions and data transformations;
• recipient validation and message delivery;
• formatting data into known templates;
• authorization checks and account changes;
• idempotent background jobs.

This is where tool calling becomes more than a feature checkbox. Function calling lets an application expose a constrained set of capabilities to the model, while the application retains control over validation and execution. OpenAI’s developer documentation describes function calling as a way for models to interface with external systems, and structured outputs are designed to help enforce a developer-defined schema rather than rely on free-form text parsing. (platform.openai.com)

The important caveat: schema-valid output is not automatically business-valid output. If a model returns a valid building_height: 900, refund_amount: 100000, or send_to_all_customers: true, your application still needs normal authorization, range checks, policy checks, and user confirmation where appropriate.

A reference architecture for model-directed execution

Teams do not need to rebuild their entire product around agents to apply this approach. Start with one costly workflow and turn its recurring steps into an execution contract.

1. Define a narrow action vocabulary

List the operations your product can reliably perform. For a 3D product, that might be create_terrain, generate_road_network, place_building, apply_style, and render_scene. For a marketing product, it might be create_audience, draft_campaign, schedule_send, fetch_metrics, and create_experiment.

Avoid exposing a vague tool such as run_any_code. Narrow tools are easier to secure, test, document, price, and monitor. They also give the model less room to wander into invalid execution paths.

2. Use a typed intermediate representation

Do not pass prose directly from the model into your production system. Convert it into a versioned structured object—the intermediate representation, or IR—that contains only supported values.

For example, a campaign IR might include campaign objective, audience ID, channel, template ID, schedule, locale, and experiment flags. A world-generation IR might include a biome, road strategy, building library, density target, constraints, and seed.

The IR becomes the real product contract. Models, prompts, and UI controls can change over time; the executor should operate against a stable representation with clear compatibility rules.

3. Validate before side effects

Validation belongs between the LLM and the action. Check field types, required parameters, enum membership, range limits, permissions, account quotas, dependency availability, and policy requirements.

For high-impact operations, add a confirmation gate. It is acceptable for an LLM to prepare a bulk-send plan; it is usually unwise for it to send millions of messages because a user vaguely said “tell everyone.” Good design distinguishes planning from commitment.

4. Execute through ordinary software

The executor should be boring. It calls the renderer, workflow engine, database, storage layer, or email API with validated parameters. It emits logs, handles known failures, and returns a typed result.

For email infrastructure, that means using the service’s documented endpoints and event webhooks rather than generating and running one-off delivery logic. Teams comparing providers should evaluate predictable operational capabilities alongside headline API features; a practical email API reference and setup guide should make the deterministic portions of sending, tracking, and error handling straightforward.

5. Let the model explain, revise, or recover

After execution, bring the model back in where language helps. It can explain why a plan was rejected, summarize a completed result, convert technical validation errors into useful next steps, or propose a revision based on the user’s feedback.

This pattern prevents the model from being the source of truth for tasks better handled by application logic, without making users learn a rigid command language.

Procedural generation makes the 3D example especially compelling

Graphyti’s shift has a natural fit in 3D because buildings and cities contain abundant structure. Roads obey connectivity constraints. Parcels have boundaries. Buildings can use repeatable footprints, floors, roof styles, windows, materials, setbacks, and placement rules. Terrain can be built from height fields, masks, erosion rules, and biome data.

Procedural tools are designed to make a compact set of parameters produce more complex output. Blender’s Geometry Nodes system, for instance, supports node groups that define operations to modify geometry, while its fields system describes calculations that can be evaluated repeatedly across geometry elements. (docs.blender.org)

That makes a strong separation possible:

The generator layer can also exploit classic performance techniques. Blender documentation notes that instancing can reference shared geometry instead of duplicating data, allowing complex scenes to reuse underlying meshes more efficiently. (docs.blender.org) A language model generating unique code and unique geometry decisions for every repeated object may miss these established optimizations; a dedicated pipeline can make them defaults.

When the upfront investment is worth it

The Graphyti team is candid about the tradeoff: building and maintaining generators is substantial upfront work, and it only pays off where the domain has enough encodable structure.

That warning matters. “Make everything deterministic” is not a sensible universal mandate. Early-stage teams can waste months formalizing workflows before they understand what customers actually want.

A useful decision framework has four questions.

Is the workflow frequent enough?

A one-off, internal task may not justify a custom engine. A task that runs thousands of times per month, sits on a core user journey, or accounts for a large share of inference spend is a stronger candidate.

Is there a stable grammar underneath the task?

Domains with objects, constraints, taxonomies, reusable steps, and accepted output formats are ideal. Examples include CAD-like generation, logistics planning, invoice extraction, ecommerce catalog enrichment, CRM updates, scheduling, campaign assembly, and analytics queries.

Can quality be measured objectively?

If you can define validation rules, benchmarks, simulations, unit tests, human-review rubrics, or business metrics, a deterministic layer becomes easier to improve. If “quality” is entirely subjective and every output is a new creative experiment, retain more generative flexibility.

Does repeatability create business value?

Repeatability is particularly valuable in regulated, customer-facing, collaborative, or high-volume products. If a wrong result affects money, access, delivery, legal commitments, customer data, or brand reputation, an auditable execution trail is worth far more than a clever demo.

Common implementation mistakes to avoid

Hybrid architectures can fail if teams treat tool calling as magic. The following mistakes are common.

Building a tool for every microscopic action

Excessive tool granularity can create long planning loops, balloon context, and make the model responsible for orchestration details that a normal workflow engine could handle. If placing one building requires fifteen independent calls, the system may have simply traded code generation for tool-call churn.

Prefer tools that represent meaningful domain operations. generate_residential_block may be a better tool than individual calls for draw_wall, add_window, add_door, and create_roof.

Allowing the LLM to choose unrestricted parameters

A schema with arbitrary strings and unbounded numbers is not much of a control plane. Restrict model outputs to enums, reasonable ranges, supported IDs, and explicit defaults.

When a user asks for something unsupported, the correct behavior is not for the model to invent parameters until the generator breaks. It should explain the constraint, offer alternatives, or ask a clarifying question.

Treating deterministic output as automatically correct

Deterministic bugs are still bugs; they are simply repeatable bugs. A faulty zoning rule can create an entire city of broken road connections just as reliably as a model can hallucinate one.

Test generators with representative inputs, property-based tests, regression fixtures, visual snapshots where relevant, and domain-specific validators. Store golden test cases from real customer requests, especially cases that previously triggered failures.

Ignoring versioning

If a generator changes, the same scene plan may produce a different output. That may be desirable for a new version, but it should be intentional.

Version the plan schema, generator implementation, asset library, and rendering pipeline. Let customers preserve an existing artifact under its original version while creating new artifacts with the improved generator.

Measuring only token savings

Lower token use is valuable, but it is not the only KPI. Track p50 and p95 latency, successful completion rate, retry rate, validation rejection rate, support tickets per thousand runs, cost variance, and customer edits after the first result.

A system that uses fewer tokens but causes users to manually repair every output is not truly more efficient.

A phased migration plan for AI SaaS teams

You do not have to abandon an all-LLM prototype overnight. A gradual migration is usually safer and provides evidence for where custom engineering will have the highest payoff.

1. Instrument the existing workflow. Measure tokens, model calls, time spent in each step, failure modes, retries, and support incidents. Segment by customer, prompt category, and output complexity.
2. Find repetitive expensive behavior. Look for boilerplate code generation, predictable transformations, repeated retrieval, formatting, API composition, and repair loops.
3. Extract one deterministic capability. Build a single function, rule engine, template, or generator for the most common high-cost operation.
4. Introduce a schema. Ask the model to populate a constrained plan rather than generate the final artifact. Validate it before execution.
5. Run the new and old paths in parallel. Compare quality, cost, latency, and customer preference on real or shadow traffic.
6. Create replayable test fixtures. Save successful and failed plans with version metadata. Every important production issue should become a regression test.
7. Expand only after evidence. Convert additional steps when the data shows stable demand and a clear structured representation—not merely because deterministic systems sound more rigorous.

The first extracted function should be small enough to ship quickly but meaningful enough to move a metric. Examples might include deterministic document formatting after LLM extraction, a pricing calculator after intent parsing, a known report query after natural-language classification, or a reusable scene component after architectural style selection.

What the r/SaaS discussion signals for AI builders

The original Reddit post did not include substantive top-comment discussion in the supplied material, so it should not be presented as a broad community consensus. Still, the post is useful because it articulates a pattern many production AI teams encounter privately: what feels like a general intelligence problem during prototyping often turns out to contain a large, structured systems problem.

The early prototype has value. It reveals the user’s language, the range of desired outcomes, surprising edge cases, and the product moments customers find magical. But once patterns emerge, teams can convert repeated behavior into software assets.

That evolution is often a competitive advantage. A competitor can copy a prompt, call a similar model, and demonstrate a flashy result. It is much harder to copy a mature library of validated generators, schemas, business rules, test fixtures, asset mappings, fallback behavior, observability, and cost controls.

In this view, deterministic systems are not merely a cost-cutting measure. They are the proprietary operational layer around a commodity model.

The bigger principle: intelligence should be bounded by product design

The most durable AI products will not necessarily be those that give models maximum freedom. They will be the products that decide carefully where freedom helps the customer and where constraints protect the customer.

Let the model interpret fuzzy requests. Let it offer ideas, identify intent, recognize exceptions, and communicate naturally. But when a product must calculate, send, build, modify records, allocate resources, enforce policy, or generate production assets at scale, give the model an explicit contract and hand execution to systems designed for the job.

Graphyti’s reported move from roughly 4.5 million tokens and 15 minutes to 1.6 million tokens and seven minutes is a concrete reminder that architecture can matter as much as model selection. The team did not necessarily make the model more capable. It made the overall system more disciplined.

For founders, marketers, and builders, the question is not “Where can we use an LLM?” It is “Which parts of this user journey require language intelligence, and which parts should be a dependable product capability?” The answer can determine whether an AI feature remains an expensive novelty or becomes a scalable business.

FAQ

What are deterministic AI workflows?

Deterministic AI workflows use an LLM to interpret intent or select from constrained options, then use conventional software to execute repeatable work. Given the same approved inputs, versions, and seed, the execution layer should produce the same result.

Why are deterministic AI workflows cheaper than all-LLM workflows?

They reduce the need for models to generate verbose code, repetitive instructions, and repair attempts. Functions, rules engines, APIs, and procedural generators typically execute known operations more quickly and with more predictable resource use than repeated inference.

Can an LLM still be creative in a deterministic system?

Yes. The model can create briefs, select styles, propose combinations, translate natural language into parameters, and suggest revisions. Determinism applies to the controlled execution of an approved plan, not necessarily to the ideation phase.

When should a startup avoid building deterministic generators?

Avoid heavy custom implementation when usage is low, customer needs are still unclear, the task lacks stable structure, or quality cannot yet be measured. Prototype with models first, instrument real behavior, and formalize only the recurring parts that justify the engineering effort.

Does structured output make an AI workflow safe by itself?

No. Structured output helps ensure that the model returns data in an expected shape, but applications still need authorization, range checks, business-rule validation, rate limits, logging, confirmations, and normal security controls before taking action.