Why the green is earned

A run ends in a report, not a claim. This page is what that report can tell you — that the checks were real, that the agent did the work, that nothing it wasn't allowed to touch was touched — how each check is made, which are always on and which you arm in the spec, and where each has honest limits. loopgen lint catches a bad spec before any agent runs at all.

What the report can tell you

"All checks passed" is only meaningful if the checks actually exercise the new requirement and the agent actually did something. The loop runner that doesn't believe the agent earns that title in layers:

Real files changed — change detection (git)

Every iteration is diffed non-destructively, via a throwaway index. The report shows a git diff --stat, and a green run that changed no files is flagged as likely vacuous. Build and runtime artifacts (logs, databases, compile caches, generated assets) are excluded so that merely running the test suite can't masquerade as "the agent did work"; add your own globs with workspace.ignore. When the workspace isn't a git repo (or is git-ignored), it falls back to an independent content-hash walk of the working tree — so a driver that omits (or fabricates) changedFiles still can't silence the vacuous-success guard; driver-reported files are only consulted as a secondary signal when the walk hits its file cap. The run then carries a persistent caveat: no unified diff, and very large trees are capped.

The checks were red before the work — baseline evaluation

The baseline runs the checks once before any agent work. If they already pass, your checks don't test the requirement — there is nothing for the agent to earn — so by default (limits.baseline: "strict", or --strict-baseline) the run fails right there with baseline-vacuous, before a single agent turn is spent. true (or --baseline) downgrades that to a warning on an otherwise-normal run. false skips the pre-agent run entirely; set it when your checks have side effects (db migrate/seed) that must not run twice. It's stack-agnostic: it just runs whatever evaluators you defined on the pre-agent workspace.

The checks didn't race each other — sequential evaluators (default)

Evaluators run one at a time (evaluation.concurrency: 1) so checks that share external state (several bin/rails checks against one SQLite database, say) can't race and deadlock into false failures. Raise evaluation.concurrency only for genuinely independent checks that are safe to run in parallel.

The success criteria weren't rewritten — spec-integrity guard

If the loop spec lives inside the workspace, the agent can edit its own success criteria. The runner watches the spec file, excludes it from the work diff (so a spec-only edit can't fake "work"), and by default (specGuard: "error") fails the run (spec-tampered) if the agent modified it, so an altered contract can't report green. "warn" lets the green stand with a caveat; "off" disables the watch. Best practice: keep specs outside the target repo.

The tests weren't edited — evaluator-integrity guard

The real success criteria for a command check are the test files it runs, and the agent could fake a green by editing them. The runner watches those files — auto-detected from the command (e.g. bin/rails test test/foo_test.rb), plus any evaluators[].guard paths — excludes them from the work diff, and by default (evaluatorGuard: "error") fails the run (evaluator-tampered) if any changed. "warn" lets the green stand with a caveat; "off" disables it. A bare runner with no file arguments (npm test) names nothing and is intentionally not watched. When both guards fire, spec-tamper takes precedence.

The agent actually finished — honest outcomes

Drivers report a stopReason (completed | max_turns | aborted | error). When the agent runs out of turns or errors but the checks pass anyway, the run still succeeds (checks are the source of truth), but the report says so instead of showing a clean green. The same honesty applies at the process level: a CLI driver whose binary fails to spawn surfaces as an error rather than a vacuous success, and a user abort takes precedence over the incidental spawn error it causes.

Linting before you run

A misconfigured spec can burn hours before failing for a reason that had nothing to do with the agent. loopgen lint catches those statically, in milliseconds, before any agent turn:

loopgen lint my-feature.loop.yaml
loopgen lint punch-list.batch.yaml      # lints the manifest + every item's spec
loopgen lint my.loop.yaml --strict      # exit non-zero on warnings too
loopgen lint my.loop.yaml --json        # machine-readable findings

It flags, among other things:

Exit codes: 2 if any errors, 1 if --strict and warnings, else 0. The error-severity workspace checks also run automatically as part of every run (the resolved workspace is printed up front); skip with --skip-preflight.

Budget ceilings

limits.maxCostUsd and limits.maxTokens (both optional) cap a run's cumulative driver-reported usage. The engine sums usage across iterations and, when a non-converging iteration pushes the total past a limit, stops with outcome budget-exceeded rather than funding another turn.

• A satisfied iteration always reports success first — getting the result is never penalized; only further spend is capped.
• An un-instrumented driver that reports no usage can never trip a budget.
• maxTokens counts input + output combined.