!bash "${CLAUDE_SKILL_DIR}/../../hooks/yaml-helper.sh" resolve_config --keys automation,workflow,qa.level,system_overrides
Resolved above — use as-is. No block → defaults in
.claude/docs/config-resolution.md.
Regression Suite
This skill ensures that every bug fix is backed by a test that would have
caught the original bug — and that the regression suite stays current as the
game evolves. It also detects when new features have been added without
corresponding regression coverage.
A regression suite is not a new test category — it is a curated list of
tests already in tests/ that collectively cover the game's critical paths
and known failure points. This skill maintains that list.
Output: tests/regression-suite.md
When to run:
- After fixing a bug (confirm a regression test was written or identify gap)
- Before a phase gate — at
qa.level: full, /gate-check requires a regression suite
- As part of sprint close to detect coverage drift
Every AskUserQuestion call follows .claude/docs/automation-modes.md
(collaborative asks always · guided major-only · autonomous logs and proceeds;
automation_always_ask categories always prompt).
Workflow tier: modes.workflow as resolved above — supplied by modes.rigor
unless set explicitly — per .claude/docs/workflow-modes.md; in audit mode consider
workflow_overrides.system_overrides.<system> per system as each GDD is read. It
sets whether GDD critical paths are mapped or coverage is smoke-only — see Step 2c.
qa.level: controls whether the suite is generated at
all. At minimal, the regression suite is not generated (report that and
stop); at standard, generate it at Polish-stage entry; at full, at
Production-stage entry. Distinct axis from workflow.
1. Parse Arguments
Early qa.level guard (resolved above): if qa.level: minimal, the
regression suite is not generated — report "Regression suite not generated at
qa.level minimal" and STOP here, before any scan, in every mode
(update / audit / report). This is the qa.level axis; it is distinct from
the workflow-tier minimal branch in Step 2c (which only changes the
critical-path source, not whether the suite runs). Do not enter Step 2c's
minimal branch on account of qa.level. This stop's verdict is
NOT ASSESSED (Section 7) — tests are not required at this level, and nothing
was scanned.
Modes:
/regression-suite update — scan new bug fixes this sprint and check
for regression test presence; add new tests to the suite manifest/regression-suite audit — full audit of all GDD critical paths vs.
existing test coverage; flag paths with no regression test/regression-suite report — read-only status report (no writes); suitable
for sprint reviews- No argument — if a sprint is clearly active (sprint plan exists with in-progress stories), run
update. If ambiguous or no active sprint is detected, use AskUserQuestion:
- Prompt: "No subcommand specified. Which mode do you want to run?"
- Options:
[A] update — scan new bug fixes this sprint and add missing regression tests[B] audit — full audit of all GDD critical paths vs. existing test coverage[C] report — read-only status report (no writes)
2. Load Context
Step 2a — Load existing regression suite
Read tests/regression-suite.md if it exists. Extract:
- Total registered regression tests
- Last updated date
- Any tests flagged as
STALE or QUARANTINED
If it does not exist: note "No regression suite found — will create one."
Step 2b — Load test inventory
Glob all test files:
tests/unit/**/*_test.*
tests/integration/**/*_test.*
tests/regression/**/*
For each file, note the system (from directory path) and file name.
Do not read test file contents unless needed for name-to-test mapping.
Step 2c — Load GDD critical paths
For audit mode: read design/gdd/systems-index.md to get all systems, then
scope the scan by each system's workflow tier (resolved above):
full — read the GDD and map critical paths from all sections.standard — same, from the required sections (Acceptance Criteria, Edge
Cases, and Formulas where the system defines numeric rules).
A system pinned higher via system_overrides is mapped at its higher tier.minimal — skip the GDD critical-path scan. Instead read the latest
smoke-check report in production/qa/smoke-*.md and take the critical paths it
exercises as the regression scope (Step 3 maps coverage against those, not GDD
acceptance criteria). If no smoke report exists, stop with Verdict: NOT
ASSESSED — no smoke report to take critical paths from — there is no
critical-path source at minimal without one; run /smoke-check first.
(Tier affects audit mode only; update and report modes are tier-independent.)
For each in-scope MVP-tier system's GDD, extract:
- Acceptance Criteria (these define the critical paths)
- Formulas section (formulas must have regression tests)
- Edge Cases section (known edge cases should have regression tests)
For update mode: skip full GDD scan. Instead read the current sprint plan
and story files to find stories with Status: Complete this sprint.
Step 2d — Load closed bugs
Glob production/qa/bugs/*.md and filter for bugs with a Status: Closed
or Status: Fixed field. Note:
- Which story or system the bug was in
- Whether a regression test was mentioned in the fix description
3. Map Coverage — Critical Paths
For audit mode only. (At minimal the critical paths come from the smoke-check
report identified in Step 2c, not from GDD acceptance criteria — map coverage
against those smoke paths and skip the GDD-criterion loop below.)
For each GDD acceptance criterion, determine whether a test exists:
- Grep
tests/unit/[system]/ and tests/integration/[system]/ for file names
and function names related to the criterion's key noun/verb
- Assign coverage:
| Status | Meaning |
|---|
| COVERED | A test file exists that targets this criterion's logic |
| PARTIAL | A test exists but doesn't cover all cases (e.g. happy path only) |
| MISSING | No test found for this critical path |
| EXEMPT | Visual/Feel or UI criterion — not automatable by design |
- Elevate MISSING items that correspond to formulas or state machines to
HIGH PRIORITY gap — these are the most likely regression sources.
4. Map Coverage — Fixed Bugs
For each closed bug:
- Extract the system slug from the bug's metadata
- Grep
tests/unit/[system]/ and tests/integration/[system]/ for a test
that references the bug ID or the specific failure scenario
- Assign:
- HAS REGRESSION TEST — a test was found that would catch this bug
- MISSING REGRESSION TEST — bug was fixed but no test guards against recurrence
For MISSING REGRESSION TEST items:
- Flag them as regression gaps
- Suggest the test file path:
tests/unit/[system]/[bug-slug]_regression_test.[ext]
- Note: "Without this test, this bug can silently return in a future sprint."
5. Detect Coverage Drift
Coverage drift occurs when the game grows but the regression suite doesn't.
Check for drift indicators:
- Stories completed this sprint with no corresponding test files in
tests/
- New systems added to
systems-index.md since the last regression-suite update
- GDD sections added or revised since the regression suite was last updated
(use Grep on GDD file modification hints if available, or ask the user)
tests/regression-suite.md last-updated date vs. current date — if gap >
2 sprints, flag as likely stale
6. Generate Report and Suite Manifest
Report format (in conversation)
## Regression Suite Status
**Mode**: [update | audit | report]
**Existing registered tests**: [N]
**Test files scanned**: [N]
### Critical Path Coverage (audit mode only)
| System | Total ACs | Covered | Partial | Missing | Exempt |
|--------|-----------|---------|---------|---------|--------|
| [name] | [N] | [N] | [N] | [N] | [N] |
**Coverage rate (non-exempt)**: [N]%
### Bug Regression Coverage
| Bug ID | System | Severity | Has Regression Test? |
|--------|--------|----------|----------------------|
| BUG-NNNN | [system] | S[N] | YES / NO ⚠ |
**Bugs without regression tests**: [N]
### Coverage Drift Indicators
[List new systems or stories with no test coverage, or "None detected."]
### Recommended New Regression Tests
| Priority | System | Suggested Test File | Covers |
|----------|--------|---------------------|--------|
| HIGH | [system] | `tests/unit/[system]/[slug]_regression_test.[ext]` | BUG-NNNN / AC-[N] |
| MEDIUM | [system] | `tests/unit/[system]/[slug]_test.[ext]` | [criterion] |
Suite manifest format (tests/regression-suite.md)
Before computing coverage, check the denominator. If the GDD
glob returns zero critical paths, or the test globs return zero test
files, do not emit a percentage — report
Coverage: NOT ASSESSED — [no GDDs found | no test files found] and name which
side was empty and the skill that produces it (/map-systems and
/design-system for GDDs, /test-setup for the test scaffold).
A percentage computed from an empty denominator is not a low score; it is not a
number. 0% reads as "measured and terrible" and 100% as "measured and
perfect" — both are claims about a comparison that never happened. This is the
same defect /scope-check carries a Phase 4 guard against, in the same words:
"a percentage computed from no baseline items is not a small number; it is not
a number."
A hand-written list of skills required to carry NOT ASSESSED pins what was
known when it was written, so a skill added later inherits no obligation and
nothing notices. Derive that set rather than enumerating it.
The manifest is a curated index — not the tests themselves, but a registry
of which tests should always pass before a release:
# Regression Suite Manifest
> Last Updated: [date]
> Total registered tests: [N]
> Coverage: [N]% of GDD critical paths
## How to run
[Engine-specific command to run all regression tests]
## Registered Regression Tests
### [System Name]
| Test File | Test Function (if known) | Covers | Added |
|-----------|--------------------------|--------|-------|
| `tests/unit/[system]/[file]_test.[ext]` | `test_[scenario]` | AC-N / BUG-NNNN | [date] |
## Known Gaps
Tests that should exist but don't yet:
| Priority | System | Suggested Path | Covers | Reason Not Yet Written |
|----------|--------|----------------|--------|------------------------|
| HIGH | [system] | `tests/unit/[system]/[path]` | BUG-NNNN | Bug fixed without test |
## Quarantined Tests
Tests that are flaky or disabled (do not run in CI):
| Test File | Function | Reason | Quarantined Since |
|-----------|----------|--------|-------------------|
| (none) | | | |
7. Write Output
In update or audit mode, ask: "May I write/update tests/regression-suite.md with the current
regression suite manifest?"
For update mode: append new entries; never remove existing entries
(use Edit with targeted insertions).
For audit mode: rewrite the full manifest with updated coverage data.
For report mode: do not write anything.
After writing (if approved):
- For each HIGH priority gap: "Consider creating the missing regression test
before the next sprint. Run
/test-helpers to scaffold the test file."
- If bug regression gaps > 0: "These bugs can silently return without regression
tests. The next sprint should include a story to write the missing tests."
- If coverage drift detected: "Regression suite may be drifting. Consider
running
/regression-suite audit at the next sprint boundary."
At workflow: minimal, which has no sprints, say "next story" for "next sprint" in the three lines above.
Verdict, first match wins:
- NOT ASSESSED — the Section 1 stop at
qa.level: minimal: "Regression
suite not generated at qa.level minimal". Tests are not required at this
level; nothing was scanned or written.
- NOT ASSESSED — the
minimal stop above: no smoke report to take critical
paths from; name /smoke-check as the fix.
- NOT ASSESSED — coverage came back
Coverage: NOT ASSESSED — [reason] (no GDDs or
no test files); name the reason. Never report COMPLETE over a coverage figure
that was not computed.
- BLOCKED —
update or audit mode and the user declined the write.
- COMPLETE —
update / audit: regression suite updated; report: coverage
report shown (report mode writes nothing).
Collaborative Protocol
Applies in collaborative mode (the default). For guided and
autonomous modes, see .claude/docs/automation-modes.md — the rules below
describe what collaborative mode requires, not universal behavior.
- Never remove existing regression tests from the manifest without
explicit user approval — removing a test that was deliberately written is a
regression risk itself
- Gaps are advisory, not blocking — surface them clearly but do not prevent
other work from proceeding (except at
qa.level: full, where phase gates require the regression suite)
- Quarantine is not deletion — tests with intermittent failures should be
quarantined (noted in manifest) but not removed; they should be fixed by
/test-flakiness
- Ask before writing — always confirm before creating or updating the manifest