Skip to content
stacks
Docs menu

Verify: the definition of done

Learn how stacks verify runs a project's recorded steps, what each outcome means, and why an agent can't quietly edit the checks.

View as MarkdownOpen in ClaudeOpen in ChatGPT

stacks verify runs the steps recorded in .stacks/manifest.json and reports one outcome. A project is done when verify reports green, which means the declared checks passed: the project installs, lints, typechecks, tests and serves the way its stack said it must. Verify doesn't judge the code beyond those checks.

Who decides when a project is done

The stack author declares the checks once, stacks new copies them into the manifest, and the agent's job is to make them pass. An agent that decides for itself when it's finished tends to stop too early, or to lower the bar when a test won't pass. The checks are the same on every machine, for the agent, for you and for anyone who clones the project later.

What verify runs

Verify runs only what the manifest lists. It never reads a stack or fragment and never consults the registry, so a project verifies the same way whether or not stacks still has the entry it came from. It finds the nearest .stacks/manifest.json by walking up from the current directory, so you can run it from anywhere inside the project.

stacks verify
Terminal

The brief shows the agent the same steps verify will run. From a sandboxed go-cli project:

| Step | Phase | Layer |
|---|---|---|
| `install` | setup | install |
| `git-hooks-install` | setup | install |
| `build` | check | build |
| `vet` | check | typecheck |
| `lint` | check | lint |
| `test` | check | test |
| `test-integration` | check | test |
| `smoke` | check | test |
| `readme-filled` | check | check |
...
.stacks/BRIEF.md

Each step runs through /bin/sh -c in the project root, with stdin closed, in its own process group. Its environment is yours minus the agent session markers (CLAUDECODE and the rest), with the manifest's env filling any key you haven't set. PORT_WEB=5187 stacks verify moves a port without touching the manifest.

Phases

Every step has a phase, and phases run in order:

Phase What it's for Failure behavior
setup Installing dependencies, starting a database The first failure stops setup and skips the check and serve steps
check Lint, typecheck, tests, file checks Every check runs, so one run reports every failing check
serve Starting the app and probing it over HTTP Each server is started, probed until it answers, then stopped
teardown Stopping what setup started Always runs, after a failure or a Ctrl+C too

Setup, check and teardown steps each have a timeout (default 10m). A step that exits 127 is reported as a missing command, with failureKind: "env", because that usually means a tool isn't installed.

A serve step names a URL to probe (http on localhost, 127.0.0.1 or ::1). Verify refuses to start the step if that port already answers, then polls the URL every 500 ms until it gets any status below 500, for up to 60 seconds by default. The server must stay in the foreground: one that exits before the probe passes fails, and so does one still answering after verify stopped its process group.

A project with an app runs in two stages. The base phases finish first, teardown included. App steps run only after a fully passing base; a failed base skips them all.

Layers

Every step also has a layer, which says what it proves:

Layer Proves
install Dependencies install
lint The linter passes
typecheck The type checker passes
test Tests pass
build The project builds
serve The app starts and answers HTTP
docs A documentation step
check A file check, such as "the README has no fill-in markers"

stacks validate requires every stack to cover install, lint, typecheck and test, plus a probed server for web and API stacks.

Outcomes

A run ends with exactly one outcome:

Outcome Exit Meaning
green 0 Every step ran and passed. The project is done
partial 0 Every step that ran passed, but some were left out on purpose with --only or --skip-tag. Never counts as done
failed 5 A step failed. The report names it, its command, its failure kind (step, env or timeout) and the end of its output
changed 5 The steps don't match the recorded copy, so nothing ran. See Tamper evidence

partial runs are for fixing failures. While an agent works on two failing checks, it can rerun only those:

stacks verify --only lint,test
Terminal

Setup and teardown still run around them. When both pass, run plain stacks verify again for a green.

--all adds steps that are skipped by default, which means prod steps in a sandboxed project. A green run without them is still green.

Note

Exit 0 alone doesn't mean done, because partial also exits 0. Scripts and agents should read the outcome: stacks verify --json returns it as outcome.

A failed run keeps its full report on stdout and exits 5. With --json, the report is one VerifyResult object on stdout and the last stderr line is the error envelope; Output, JSON and exit codes has the full contract.

Tamper evidence

Verify refuses to run steps that changed after stacks new recorded them, so an agent can't get a pass by editing a failing test step to true.

When new writes the project, it records a digest of the verify steps, the manifest's env and its preset, in two places:

  1. verifyDigest in .stacks/manifest.json, inside the project.
  2. The project's state record, outside the project, at ~/.local/state/stacks/projects/<projectId>.json (or under STACKS_STATE_DIR).

Before every run, verify recomputes the digest and compares it with both copies. If either disagrees, nothing runs, not even teardown. The outcome is changed, the exit is 5 (VerifyChanged), and the report says which copy disagreed and which steps were edited, added or removed, or whether env or preset changed.

Editing project files can't hide the change, because one copy lives outside the project. The brief tells the agent what to do on changed: stop, tell you which steps changed, and wait.

Accepting a change on purpose

To change the steps on purpose, such as adding a test target or raising a timeout, make the edit and run this at your own terminal:

stacks verify --accept-changes
Terminal

It prints what changed, records the new digest in the state record and then the manifest, and runs the steps. It's refused with PermissionDenied (exit 6) when an agent is detected or stdin isn't a terminal, and then changes nothing.

Verifying a fresh clone

A fresh clone has no state record on that machine, so verify checks the manifest's copy only and reports tamperCheck: "manifest-only". That still catches an edit made without updating verifyDigest, but not one where both were changed together. Review manifest diffs in pull requests the way you'd review any CI config.

Important

The digest catches edits to the steps. It can't catch an agent that makes a step pass by weakening the code it runs: an empty test target, a lint config that ignores everything, deleted tests. The brief forbids all of those, and they show up in the diff. Read the diff before you trust a green.

Run records

Every run that gets past the manifest, including changed, failed and interrupted ones, appends one line to .stacks/state/runs.jsonl: start time, CLI version, detected agent, outcome, scope, tamper check, durations, and each step's name, phase, layer, status, failure kind and exit code. Records hold no commands, output, paths or environment values, so one is safe to attach to a bug report.

The state record also keeps two timings that are never overwritten: firstGreenAt, set by the first green run, and firstTestPassAt, set by the first run in which a test-layer step succeeded. stacks runs summarizes the history:

stacks runs
Terminal