<!-- Generated by forge sync. -->
Test Audit
Two modes, one value bar. Authoring mode gates every new or changed test at
write time. Audit mode runs focused sweeps of tests that re-assert source,
duplicate stronger proof, couple behavior to implementation, or keep test-only
production seams alive. Continue broad audits as separate coherent Forge tasks
or fixes; optimize for confidence, not deletion count.
A Forge task names the tests it adds or changes in its story doc's Tests
column, and forge.toml's test command runs the whole suite. For Forge's own
repo, tests live under tests/; a client repo uses the paths its story doc
names. forge close <item> owns the review and the checks.
Authoring gate
Before adding or changing any test, answer four questions; a missing answer
means do not add it yet:
- What observable behavior, invariant, or independent contract does it
protect?
- What credible regression makes it fail?
- Why does existing coverage not already catch that failure? Each contract has
one primary test owner at the strongest boundary; another layer needs its
own distinct risk, such as a transport or lifecycle failure the owner cannot
reach. Prefer extending a table-driven case or shared fixture over a
near-duplicate test; consolidate duplicated setup in the same change.
- Does it need a production seam (export, flag, wrapper, injection hook) that
no production caller needs? If yes, move the test to the real boundary
instead.
Then check the test against every junk pattern; a match fails
the gate unless the retention bar names the contract it
independently guards. A test that would break under behavior-preserving
refactoring is asserting implementation, not behavior; rewrite it at the
owning boundary before landing it.
Bug regression tests must fail on the pre-fix code for the intended reason and
pass after the owner-boundary repair. A regression test that never demonstrably
failed proves the mock, not the fix. One regression at the owner boundary
covers the bug; do not replay the same scenario at every layer it crosses.
Junk patterns
The shared checklist for authoring and audit: the gate rejects a new or changed
test that matches one, and audits hunt for existing tests that do.
- assertion-free coverage probes;
- self-comparisons and identity copiers;
- copied fixtures, inventories, manifests, or export lists;
- exact source, import, or string greps;
- private predicate or call-shape tests duplicated at real boundaries;
- duplicate invocations of the same contract;
- provider-local replays of shared helpers;
- tests whose only purpose is preserving test-only exports, globals, or
wrappers;
- dead production code whose only callers are tests;
- expected values produced by the helper or renderer under test;
- mocks that implement the asserted behavior, or one identical mock standing in
for different APIs;
- fixtures that supply the receipt, admission, or callback ordering the owner
should produce, or persistence asserted against a store the path never
writes;
- capability tests that restate declared flags instead of exercising the
delivery or acknowledgement the flag promises;
- negative controls that pass for an unrelated reason, such as a denial from a
different guard or a rejection the production path never reaches;
- names or fixtures that promise more than the input exercises, such as a
"retires the window" test asserting the window was not cleared.
Retention bar
Tests justify their maintenance cost by protecting behavior, a credible
regression, or an independently meaningful contract. In an audit, an existing
test that must change for behavior-preserving source reorganization is suspect,
not automatically deletable; the authoring gate still rejects new ones.
Before judging a candidate, read the complete test and production owner, its
entry point, callers, callees, sibling implementations, overlapping tests, CI
routing, and relevant history. Read root and scoped AGENTS.md files first.
When the test claims dependency-backed behavior, inspect the dependency source
or types directly.
Keep a test when it independently enforces a public API, SDK, protocol, config,
migration, storage, security, platform, default, prompt-byte, generated
cross-language, package, release, or architecture contract. Also keep:
- call ordering when order is observable behavior;
- regressions with a credible failure mode;
- source inspection when it is the cheapest independent guard: it fails when
the contract changes (the user-facing key, byte, or path) and survives an
identifier-only refactor;
- a retained test that fails on the baseline: treat it as a possible product
bug, reproduce it, and repair the owner rather than deleting it.
Static or slow is not a deletion reason. A test that resembles implementation
may still be the independent contract; prove otherwise before removing it.
Discovery
Keep discovery read-only and report evidence before editing. For broad scope,
organize discovery around production owner boundaries where the story's tasks
and Scopes permit parallel work. Include relevant tests at shared core
boundaries and QA or live-proof harness tests. For Forge itself, inspect
tests/; for client work, use the test paths the story doc names. Prefer a
few high-confidence candidates over a large speculative inventory. Hunt for the
junk patterns.
Candidate evidence
Record every field below before editing. A missing field means the candidate
is not ready for deletion:
- exact test name and location;
- what failure it can actually detect;
- non-test callers of the covered production or support seam;
- stronger remaining owner-boundary proof, or why no proof is needed;
- relevant history and the reason the test or seam exists;
- production or test-support deletion unlocked;
- risk and the focused validation command.
Keep the evidence in the task's notes or the commit message; never hand-write
anything under .factory/, which only forge commands write.
Edit shape
Choose one coherent owner-boundary batch. Delete obsolete test-only exports,
globals, wrappers, and dead production paths instead of preserving aliases.
Move retained regressions to their canonical owners. Consolidate repeated
package or dependency assertions into one generic contract.
Prefer net-negative production LOC. Do not add replacement tests that restate
the same implementation, and do not convert uncertain candidates into cleanup
to increase deletion counts.
Validation
Never edit source or tests while a test suite is running in the checkout.
Follow forge.toml's test_place. In local mode, use forge test for validation.
In CI mode, do not run them here for a passing
result; run forge test only for a test you are writing, to watch it fail.
Pull request CI runs the change's related tests, lint and type checks.
- Identify the smallest owner and sibling tests the task names in its Tests
column; include the relevant
tests/ files when changing Forge itself.
- For source greps or plan assertions, name the executable contract owner;
source inspection alone is not test evidence.
- Before you stop, commit your work first. In local mode, run the change's related
tests through
forge test in the foreground and wait for it to finish. Run tests
only through forge test: it runs forge.toml's fast_test, or test when none
is set, in the machine's one test lane, with {base} as the merge base with
origin/<default branch>. It always runs, including uncommitted changes. In CI
mode, commit your work and leave the change's related tests to pull request CI.
Then commit any fixes from the local run. CI runs the full suite.
forge close <item> owns the review and the checks. Do not reconstruct its
review or check wait with ad hoc commands.
- After an audit batch, inspect the diff and report production/tooling changes
separately from tests and test support.
Landing and continuation
Complete work only within the task's Scope. Each Forge task or fix owns its
worktree, review, and pull request; a human merges. Keep one coherent
owner-boundary batch per task. Continue broader audits through separate tasks
or fixes, refreshing discovery against the current default branch before each
batch.
Handoff
Report:
- root cause and removed low-value categories;
- production owner simplifications;
- retained false positives and why they remain valuable;
- focused and full proof actually run;
- production versus test LOC;
- task and pull request state;
- named follow-ups.