AI Agent Verification: How Unlazy Makes “Done” Provable

AI agent verification is becoming a core engineering problem: coding agents can produce impressive output, but a confident final message is not the same thing as a completed, tested change. Unlazy, an open-source skill from Leon Lin’s GitHub account, takes a refreshingly concrete position on that gap: define what success means before work begins, execute only checks a human has reviewed, and report completion only when the evidence supports it.

The project was highlighted in a YouTube walkthrough as a tool for preventing the familiar failure mode of long AI-assisted coding sessions: an agent says it implemented and tested a substantial task, but the code contains placeholders, a requested requirement disappeared, or the tests were never actually run. The video’s central insight is worth taking seriously even beyond this one tool: the biggest reliability upgrade may not be a better “finish the task” prompt. It may be a better system for deciding when a task is truly finished.

The real AI coding problem is not generation — it is completion

AI coding agents are increasingly capable of editing repositories, running commands, opening pull requests, and working through multi-step tasks. But the more independent the workflow becomes, the more important it is to distinguish between activity and verified completion.

An agent can create many files, explain its choices clearly, and still fail the original job. It may implement the happy path but omit errors and migrations. It may write a test file without running it. It may make a change that looks locally correct while breaking a neighboring package. And because language models are optimized to produce plausible continuations, their final summaries can sound more certain than the underlying state of the repository warrants.

That failure pattern becomes more likely as work gets longer and more iterative. SlopCodeBench, a 2026 research benchmark focused on agents extending their own code through evolving specifications, reported a best tested performance of 14.8% across its checkpoints and no fully solved problem in its evaluated setting. That does not measure Unlazy, nor does it prove every real-world agent run will fail. It does reinforce the underlying concern: single-shot coding demos can hide the degradation that appears when requirements evolve and an agent must preserve its earlier work. (arxiv.org)

METR’s task-completion-horizon research offers another useful framing. It defines a time horizon as the human-expert task duration at which an agent reaches a specified reliability level, rather than treating capability as a binary label. In plain language, a model may be excellent on a task that takes a person ten minutes while becoming much less dependable on a task that takes several hours. That makes process controls more valuable precisely when teams hand agents larger refactors, migrations, audits, and cross-package changes. (metr.org)

Unlazy’s premise is therefore not that an agent is useless without extra structure. It is that a model should not be the final authority on whether its own work satisfies a specification.

What is Unlazy?

Unlazy is an MIT-licensed, open-source skill for substantial AI-agent work. Its repository describes the core workflow in four parts: write an acceptance ledger first, execute reviewed checks, reverify returned work, and report only what the evidence supports. The currently published source targets version 2.1.0, though the repository advises teams that need immutable installs to pin an exact commit rather than assume an untagged source state will remain unchanged. (github.com)

The skill is designed for coding-agent environments that consume SKILL.md-style instruction bundles, including Claude Code and Codex-based workflows. That positioning is increasingly relevant because skills have become a common way to package reusable procedures, conventions, and multi-step workflows for agents. OpenAI describes skills as versioned bundles containing instructions and a SKILL.md manifest, compatible with the open Agent Skills standard. (developers.openai.com)

Unlazy is not a test framework, a sandbox, or a replacement for code review. It is better understood as a completion protocol layered around the commands and checks a repository already uses.

Its main mechanism is a Markdown file, usually called GATES.md or gates.md, that acts as an acceptance ledger. Every meaningful outcome is written down as a gate. Where an outcome can be mechanically checked, the gate includes:

• a CHECK: command that can be executed in a shell;
• an EXPECT: marker that must appear in the combined command output;
• evidence recorded after a successful run; and
• an explicit abandonment record when an originally required gate cannot be met.

A gate is not complete merely because the agent checked a box. It counts as met only when the command exits successfully and the expected output is observed. That dual requirement is a small but important design choice: an exit code of zero can be misleading, while a printed success phrase alone can be faked or emitted too early.

The acceptance ledger changes what “done” means

The most useful idea in Unlazy is not the command runner. It is the shift from a conversational definition of done to an artifact-based definition of done.

A conventional agent task often looks like this:

1. A human writes a broad request in chat.
2. The agent interprets the request and modifies code.
3. The agent reports what it believes it completed.
4. The human inspects the diff and tries to determine whether the report was accurate.

An acceptance-ledger workflow changes the order:

1. Translate the request into observable outcomes.
2. Decide how each outcome will be verified.
3. Review those verification commands before allowing them to run.
4. Implement the work.
5. Run the checks and attach evidence.
6. Re-run the checks at handoff time.
7. Report only outcomes that survived verification.

This is essentially an agent-friendly version of testable acceptance criteria. It is familiar to disciplined software teams, but Unlazy makes the criteria explicit, executable, and tied to an agent’s stop condition.

A simple gate example

For a feature that adds an account-deletion endpoint, a weak gate might be:

- [ ] G1: Account deletion works

That title is understandable to a person, but it provides no reliable proof. A stronger gate could be:

- [ ] G1: Account deletion removes the user and invalidates active sessions
  CHECK: pnpm test --filter account-deletion -- --runInBand
  EXPECT: account deletion verification passed

The stronger version still depends on the quality of the underlying test. If the test is shallow, the gate is shallow. But it makes the verification method visible before the agent starts coding, and it gives the reviewer something specific to challenge.

That matters because the checker can verify only the declared oracle. It cannot prove that a gate title such as “securely deletes accounts” matches what the command actually tests. Unlazy’s documentation acknowledges this limitation directly and encourages checks that inspect real artifacts, include independent assertions, and print a success-only marker only after all assertions pass. That intellectual honesty is one of the project’s best qualities. (github.com)

Why EXPECT: is more than ceremony

Requiring a textual success marker might initially seem redundant. If a command exits with status zero, why inspect its output too?

Because real verification scripts are imperfect. A script may swallow a failure, return success after running only setup, or report an ambiguous status. Requiring a distinctive success marker creates another condition for passing. It also encourages authors to write verification scripts with a clear terminal assertion rather than relying on incidental command output.

The caveat is crucial: EXPECT: does not make a weak command strong. A test script that prints “verification passed” without measuring the right thing remains weak. The ledger is a discipline for exposing and reviewing the proof, not magic that transforms any shell command into a trustworthy test.

Reviewed execution is Unlazy’s security differentiator

A great deal of agent-tool discussion focuses on whether the model can call a terminal. Less attention goes to the fact that terminal commands are code, and a GATES.md file from an unfamiliar repository can be an execution vector.

Unlazy’s answer is an approval model. In its report-only mode, the tool parses the ledger and identifies status without executing gate commands. On a first real run, pending commands are shown rather than silently executed. The user is expected to inspect the resolved command, expectation, working directory, shell, and relevant context; only then can they explicitly approve execution.

The project stores approval records outside the repository by default. Those records are bound to more than a broad gate name: the documented binding includes the absolute ledger and gate, exact CHECK: and EXPECT: content, resolved shell and working directory, timeout and limits, platform, and inherited PATH. If one of those inputs changes, approval needs to be renewed. (github.com)

That design protects against a simple but serious failure case. Imagine you approved a harmless command yesterday:

pnpm test --filter api

Then a pull changes it to something destructive, or swaps a script that the command invokes. A loose “this gate was approved once” system could run the modified command under the old approval. Unlazy instead treats the revised command context as a new approval decision.

Consent is not containment

This is where teams should be precise. Approval does not sandbox a command. Once a user approves execution, the process still runs with whatever permissions, credentials, filesystem access, network access, and shell authority are available in that environment.

The practical rule is simple:

• Do not approve a command you have not read.
• Do not treat a ledger inherited from an untrusted repository as safe by default.
• Inspect scripts called indirectly by the check command, not only the top-level line.
• Use disposable environments, limited credentials, and ordinary dependency hygiene for high-risk work.
• Keep the approval flow in place even if an agent claims a command is standard or harmless.

This is not an indictment of Unlazy. It is a healthy correction to the tendency to confuse a confirmation prompt with an isolation boundary.

Solo workflows: where the tool is easiest to adopt

For a single agent handling one coherent engineering task, Unlazy can be used without the more elaborate orchestration system. The basic flow is deliberately procedural:

1. Create a ledger from the project’s gate template.
2. Convert the requested outcomes into discrete gates.
3. Add runnable checks and expected success markers where possible.
4. Run a status-only report to see what is outstanding without executing commands.
5. Review the commands and approve the exact commands you consent to run.
6. Let the agent implement the task.
7. Run verification again, including a final re-verification before handoff.

The final step is important. Old evidence proves that a check passed at some earlier time; it does not prove that the repository remains correct after later edits. Reverification is the difference between “the agent once ran a test” and “the current state passed this test immediately before it reported completion.”

For a founder or small product team, this can be useful for work such as a billing refactor, a major authentication change, a production bug investigation, an SDK upgrade, or an analytics event audit. In each case, the human’s goal is not to micromanage every keystroke. It is to agree on what evidence would make the result credible.

When a manual ledger is worth the overhead

The ledger is most valuable when quiet incompleteness would be costly. Good candidates include:

• refactors with several behavior-preservation requirements;
• migrations that need both forward and backward compatibility checks;
• changes that cross frontend, API, database, and deployment boundaries;
• security-sensitive flows such as permissions, billing, deletion, or authentication;
• release-readiness fixes where several regressions must be ruled out; and
• tasks delegated to an agent for long autonomous runs.

For a typo, one-line style change, or quick exploration, the process is probably overkill. Unlazy itself positions the workflow around substantial autonomous work, not every tiny edit. The right question is not “Can I use gates?” but “Would a false claim of completion here cost more than writing the gates?” (github.com)

AI agent verification at scale: depth trees and parallel work

The project becomes more ambitious when a task is large enough to split across multiple work streams. Its orchestration approach uses a “Depth Tree” that breaks a top-level request into coherent leaves, with each leaf owning a narrow deliverable and its own verification ledger.

Before dispatching those leaves, the workflow calls for a plan that fixes interfaces, dependencies, conventions, and file ownership. That is a practical response to a common multi-agent problem: agents can work in parallel only until they collide over the same files, assumptions, or API contracts.

In Unlazy’s parallel model, each leaf declares OWNS: paths describing the repository-relative files it is allowed to modify. Concurrent leaves should not own overlapping paths, and the system can claim those paths before work begins. The idea is less about bureaucratic control and more about preventing two agents from making incompatible changes to the same component.

Verification flows upward, not just outward

A key detail is that a parent does not simply accept a child agent’s claim that its leaf is finished. It re-runs the child’s gates. Branches then add integration gates for interface compatibility, end-to-end behavior, and regression checks before the root task can be considered complete.

That is closer to how a careful engineering lead should coordinate a parallel implementation:

• A leaf proves its own local contract.
• Its parent checks that the proof still holds in the integrated repository.
• Higher levels check the interactions that local tests cannot see.
• The final report waits until the root-level gates are satisfied.

This is substantially better than a generic “spawn five agents” instruction. Parallelism can shorten elapsed time, but it also multiplies the number of boundaries where unverified assumptions can slip through. A tree without integration verification merely makes those failures happen faster.

Avoid fake decomposition

The project’s documentation reportedly warns against inventing empty hierarchy just to satisfy a requested depth. That matters. A task should be decomposed at real domain boundaries—database migration, API contract, interface, test infrastructure, documentation—not into arbitrary fragments that create more coordination than value.

A useful rule for teams adopting this model: each leaf should be able to answer three questions clearly.

1. What artifact or behavior does this leaf own?
2. Which paths may it change?
3. Which command proves it fulfilled its contract?

If those answers are vague, parallel dispatch is premature. First improve the plan, then delegate.

The optional stop hook solves a behavioral problem

Some coding agents have a strong conversational tendency to wrap up. They may summarize partial progress as a final answer, especially after a difficult test failure, a context-heavy investigation, or a lengthy series of tool calls.

Unlazy includes an optional Claude Code stop hook intended to counter that behavior. When the session ledger still shows unchecked or unsupported gates, the hook can block the agent from ending the turn as though the task were complete. The project also includes an escape mechanism after repeated blocks without ledger progress, designed to prevent endless looping. The hook does not itself execute verification commands, and the project says it should not be installed without user consent. (github.com)

The bigger lesson is useful even for teams that never install the hook: completion criteria should affect the agent’s control flow, not just appear as a reminder in a prompt. If “do not stop until all gates pass” is only prose, it is easy for the model to ignore or reinterpret under pressure. If unmet gates are visible in the workflow state and can block a final completion claim, the process has real teeth.

Still, hard enforcement should be applied carefully. An agent can be unable to close a gate because the environment is broken, credentials are missing, the specification was contradictory, or a third-party dependency is down. That is why explicit abandonment with a non-empty reason is healthier than silently deleting the requirement—or forcing the model into a futile loop.

How Unlazy compares with ordinary tests, prompts, and CI

Unlazy is not a competitor to every reliability practice. It occupies the space between agent instructions and the engineering systems that already validate software.

The best implementation is usually a combination. A good Unlazy gate should often call the project’s actual test runner, type checker, linter, build command, migration verifier, browser test, or security scan. CI can then rerun the same checks in a clean environment. Human reviewers can inspect whether the gates were meaningful and whether the code meets requirements that remain difficult to automate.

In other words, the ledger should not become a parallel, improvised testing universe. It should be an auditable index of the evidence that matters for this specific agent task.

A practical adoption playbook for teams

The easiest way to get value from Unlazy is not to create a massive verification taxonomy on day one. Start with one high-risk, bounded task and learn where the friction appears.

Start with four to six outcome-based gates

Do not write gates around implementation steps such as “create a controller” or “edit the schema.” Write them around outcomes that a user, reviewer, or system can observe.

For example, an ecommerce team migrating checkout logic could begin with gates such as:

• Existing checkout sessions still complete successfully.
• New tax calculation is applied only in the intended regions.
• Refunds continue to reconcile against original payment records.
• The migration runs successfully on a representative database snapshot.
• Existing checkout regression tests pass.
• The production build and type check succeed.

That list creates a shared contract without dictating every line of implementation. It also reveals missing requirements early. If no one can state how to verify the tax calculation, the product requirement itself may need clarification.

Prefer independent checks over self-referential evidence

A common anti-pattern is using an output value as proof of itself. For instance, if a script reads a configuration variable and prints it, an EXPECT: check that looks for the same number says little about whether the configuration is correct.

Better verification measures the result independently. If an API rate limit should be 100 requests per minute, run a focused test that makes requests under controlled conditions and demonstrates the expected boundary behavior. If an agent adds a database index, query the actual database plan or schema metadata rather than checking whether the migration file contains the string CREATE INDEX.

Treat gate authoring as a design review

The moment before implementation is a high-leverage time. Ask:

• Is every must-have requirement represented?
• Does every gate test a meaningful outcome rather than a superficial artifact?
• Are the commands safe to run in this environment?
• Which important risks cannot be automated and therefore need a human review gate?
• What should happen if a gate cannot be met?

Those questions are valuable whether the implementer is an agent, a staff engineer, or a mixed team.

The project’s research posture is unusually responsible

AI tooling repositories frequently cite benchmarks to imply guaranteed gains that have not been independently demonstrated. Unlazy’s research materials take a more careful line.

The repository connects its design to research on long-horizon agent performance and premature completion, but it does not claim that those papers prove a fixed percentage improvement from using Unlazy. It also distinguishes historical internal comparisons from a defensible benchmark result when the raw artifacts are not published. Its validation protocol calls for preregistration of the exact tool commit, task prompts, model and permissions, environment, timeout rules, scoring rubric, and multiple independent runs. (github.com)

That is the appropriate standard. A tool can be conceptually strong and still need controlled evidence before anyone should claim that it makes a particular model “X% more reliable.” For teams evaluating it now, the practical metric is their own before-and-after experience:

• How often did an agent report completion with a failing requirement?
• How many defects were caught by the ledger before review?
• How much additional setup time did gates require?
• Did the structure reduce reviewer time on large tasks?
• Did the approval process prevent risky or surprising command execution?

The results will differ based on repository maturity, test coverage, task type, model, permissions, and the people writing the gates.

The trade-off: more rigor means more ceremony

Unlazy’s strongest feature is also its cost. Writing acceptance gates, reviewing commands, maintaining ownership boundaries, collecting evidence, and reverifying at the end all take time.

For trivial work, that overhead can exceed the value of the change. A team that requires a formal ledger for every wording tweak will create process fatigue and eventually route around the tool.

For large autonomous tasks, however, the calculation can reverse. Consider a three-hour agent-led migration that touches several packages and looks successful until a reviewer discovers an unhandled rollback path. Spending 15 minutes creating good gates may be far cheaper than an afternoon of diagnosis, rework, and reduced confidence in the agent workflow.

The correct framing is not “Unlazy makes AI coding slower.” It is “Unlazy trades some upfront planning and verification for fewer unsupported claims of completion.” That trade is attractive when the cost of being wrong is high.

Bottom line: make the agent’s evidence reviewable

Unlazy is compelling because it attacks a real weakness in agentic software development without pretending to solve everything. It does not guarantee that a model understands the product, writes elegant code, or chooses complete tests. It does force a more useful discipline around claims of completion.

For creators and founders, the immediate takeaway is simple: stop treating an agent’s final summary as the main deliverable. Require a small, readable set of acceptance criteria and evidence that the current repository state meets them.

For engineering teams, Unlazy is most promising as a workflow for substantial refactors, migrations, audits, and parallelized builds—especially when existing tests and CI can serve as the commands behind the gates. Its external approval model and re-verification requirement are especially sensible for a world where agent skills are increasingly installed from repositories and granted terminal access.

The tool’s larger contribution may be conceptual. In agentic development, “done” should be a claim that can be challenged, reproduced, and reviewed—not a sentence the model happens to generate at the end of a long chat.

FAQ

What is AI agent verification?

AI agent verification is the process of checking an agent’s claimed output against observable evidence, such as tests, builds, security scans, API checks, or artifact inspections. It matters because a polished natural-language summary does not itself prove the implementation is complete or correct.

Is Unlazy a replacement for CI or code review?

No. Unlazy can call existing tests and checks, then organize them around a specific agent task. CI provides repeatable automated validation, while human review remains necessary for product judgment, architecture, security reasoning, and requirements that are difficult to automate.

Can Unlazy safely run checks from any repository?

No tool should be assumed safe merely because it offers approvals. Unlazy’s approval system makes commands reviewable and binds consent to a detailed command context, but approved commands still run with the permissions available in the local environment. Review commands and the scripts they invoke before granting approval.

When should a team use an acceptance ledger?

Use one when a task is substantial enough that missing requirements, skipped tests, or unsupported completion claims would be expensive. Strong candidates include migrations, complex bug fixes, security-sensitive changes, multi-package refactors, and parallel agent workflows.

Does Unlazy prove that a feature meets the full business requirement?

Not by itself. It proves only what the declared checks actually test. The quality of the gates remains crucial: a weak or misleading command produces weak evidence, which is why teams should review the acceptance ledger as carefully as they review the implementation.