Debugging a run

When a loop run goes wrong — it fails, stalls at the iteration budget, errors out, or comes back green but suspicious — the fault is almost always in the spec or the workspace, not the agent. The engine already names the failure precisely (the outcome field) and attaches honest warnings even to green runs. This page turns that vocabulary into a diagnosis and a concrete fix, confirming the root cause without re-spending agent budget wherever possible. (If you use Claude Code, the in-repo debug-loop skill runs this diagnosis for you.)

The mindset

• The checks are the contract. A run "passes" only if its checks actually exercise the requirement and the agent did work. A success outcome that carries warnings is a finding, not a pass — treat it as suspect.
• Reproduce cheaply. Most root causes (a wrong test command, a missing binary, a vacuous check, a bad working directory) are confirmable with loopgen lint plus running the check by hand — zero agent turns. Do that before any re-run.
• Don't re-run blindly. A full loopgen run spends real budget and needs credentials. Earn the re-run with static and manual evidence first.

1. Gather the evidence

A run leaves evidence at three depths — outcome, iteration, and the agent's inner trajectory. Collect the deepest layer you have before guessing at a shallower one:

• Best: the trace. If the run was launched with --trace trace.jsonl (or a jsonl/otlp observer in the spec), you have the trajectory layer: every model message, tool call, and tool result per iteration, plus trust-guard warnings as signal records. This is the only evidence that shows what the agent did, not just what it produced. See Observing runs.
• The --report <file> JSON. If the run was launched with --report run.json, read that file — it has every iteration, each evaluator's feedback/error, the agent's stopReason/summary/error, changedFiles, the diff stat, and the run-level warnings.
• Otherwise: the terminal output. The lines that matter are outcome: <outcome> — <reason>, the baseline: line, each iter N: line, and the trailing ⚠ warnings: block (which prints even on success).
• If there's no report and the failure is cheap to reproduce — a preflight failure, an error, or a baseline-vacuous, none of which invoke the agent — re-run with --report. If it's a max-iterations failure, prefer reading what you already have over paying the budget again.

Also note the resolved workspace the engine printed (workspace: <path>): several failures are simply a wrong path.

2. Classify by outcome

The outcome field is the spine of the diagnosis:

3. Drill into a max-iterations failure

This is the real debugging case. Open the last iteration and read three things:

1. Which evaluator never passed, and its feedback/error. That's the check that blocked the loop.
2. The agent's stopReason:
   • completed — the agent decided it was done, but a check still fails. The check or the requirement is wrong, not the budget. Run the check by hand (step 5); fix the command, loosen the criterion, or sharpen requirements.
   • max_turns — the agent ran out of turns before finishing. Raise driver.options.maxTurns, and set driver.options.resume: true so it continues the same session.
   • error (with agent.error set) — a backend or credential failure, not a code problem. Read agent.error and fix the environment.
3. Did the agent change files? If changed/changedFiles shows edits but the wrong check fails, the agent may be regressing one thing to fix another — tighten requirements ("don't break X").
4. With a trace: read the blocked iteration's trajectory. The agent.event records distinguish failure shapes the report can't: the same tool call failing every turn is an environment problem; the agent editing the right file while the wrong check fails is a requirements problem; long tool-free stretches of model output are the agent talking itself into a decision the prompt should have made. Re-running with --trace costs one iteration budget — often cheaper than another blind spec tweak.

4. Investigate a suspicious success

If success: true but warnings[] is non-empty, do not declare victory. The two warnings that matter most:

• "changed no files" — the loop went green on an untouched workspace, so the checks are almost certainly vacuous. Confirm by running the check by hand on a clean checkout (step 5); it should fail before any work. If generated artifacts masked a real diff, add their globs to workspace.ignore.
• "the agent did not complete" — the green rests entirely on the checks while the agent ran out of turns or errored. Verify the checks are actually sufficient; consider raising maxTurns.

5. Reproduce the root cause (no agent budget)

Confirm the diagnosis with cheap, deterministic signals. This is what separates a guess from a diagnosis — a missing binary, a wrong working directory, a flaky test, or a check that passes on untouched code all show up here in seconds.

# 1. Static config the run may have only warned on:
loopgen lint <spec> --strict

# 2. The actual failure of a check, in the resolved workspace:
cd <resolved workspace> && <the evaluator's command>   # read the exit code + output

6. Map the root cause to a fix, then re-verify

Then re-verify in the same cheap order — re-lint → re-run the check by hand → and only then a full run:

loopgen run <spec> --strict-baseline --report run.json

That last step costs budget and needs credentials; don't launch it until the static and manual checks look right.

Preflight failures

Preflight runs the error-severity workspace checks before any agent turn (the same rules loopgen lint reports). Map each bullet in the reason / preflight.errors to its fix:

Guardrails

• Diagnose from evidence, not vibes. If you haven't read a failing check's output, you haven't found the cause yet.
• Distinguish backend failures from spec bugs. agent.ok === false / stopReason: error is an environment problem; don't "fix" the spec for it.
• A warning on a green run is a finding. Surface it; don't report the run as clean.
• Prefer the cheapest reproduction. Lint and a manual check command beat a budget-spending re-run every time.

For the full LoopReport field map, every stopReason, and the complete warning glossary, see the in-repo debug-loop skill's reference.md. The source of truth is the code: src/core/engine.ts (outcomes, warnings, report shape), src/cli/run.ts (terminal formatting), and src/drivers/types.ts (stop reasons).