無料GitHub で公開

test-strategy

Coverage-design method for qa-engineer — pyramid ratios per archetype, equivalence/boundary/property case selection, mutation score as the real coverage signal, and a flake-quarantine policy. Turns "coverage is 90%" (a number with no method) into a defensible test plan. Emits TEST-STRATEGY-{slug}.md, which the QA gate checks exists.

avelikiy/great_cto

インストール方法を見る

含まれるファイル（1）

SKILL.md4.0 KB

SKILL.md（原文）

インストールする前に、エージェントに与えられる指示の中身を確認できます。

test-strategy

Coverage % is a number with no method. This skill makes the QA plan defensible: what to test, why that set, and a real signal that the tests bite.

qa-engineer emits docs/qa-reports/TEST-STRATEGY-{slug}.md capturing the four decisions below; the QA gate checks the file exists for medium/large features.

1. Pyramid ratio — pick by archetype (not by reflex)

The right unit:integration:e2e mix depends on where the risk lives.

Archetype	unit	integration	e2e	why
library / devtools / cli	80%	15%	5%	logic-dense, few I/O seams
web-app / saas	60%	30%	10%	request→db→render seams dominate
commerce / fintech / marketplace	50%	35%	15%	money paths need cross-component proof
data-platform / streaming	45%	45%	10%	correctness lives in pipelines, not units
ai-system / agent-product	50%	20%	10% + evals 20%	behaviour is the contract → eval set (see [[decision-eval]])

State the chosen ratio in TEST-STRATEGY and justify any deviation.

2. Case selection — equivalence / boundary / property

Don't enumerate inputs; partition them.

Equivalence classes — one representative per class of behaviour (valid, invalid, empty, max). Testing 5 valid ids ≠ testing 5 classes.
Boundaries — the bug lives at the edge: 0, 1, n-1, n, n+1, off-by-one, empty, overflow, the threshold itself.
Property-based — for pure/transform logic, assert invariants over generated input (round-trip decode(encode(x))===x, idempotency, ordering) instead of hand-picked cases. Use fast-check / Hypothesis where the logic warrants it.

3. Mutation score — the only coverage that proves the tests bite

Line/branch coverage proves code ran, not that a test would fail if it broke. Mutation testing (Stryker / mutmut / cargo-mutants) flips operators/conditions and checks a test catches it. A "90% covered" module with 30% mutation score has assertion-free tests. Target: mutation score ≥ 60% on changed logic-dense files; report it in TEST-STRATEGY when the change is logic-heavy. Coverage % is the cheap proxy; mutation score is the truth.

4. Flake quarantine — a flaky test is worse than no test

A test that fails ~randomly trains the team to ignore red. Policy:

A test that fails non-deterministically is quarantined (skipped + tracked in a Beads bug), not left to rot the signal.
Quarantine is time-boxed: a bug with an owner and a deadline, not it.skip forever.
Root-cause before re-enabling (shared state, timing, real network, ordering).
This is the same discipline the eval runner needs (see the eval flaky flag in tests/eval/runner.mjs — stddev > 0.1 is a flake signal).

Output: TEST-STRATEGY-{slug}.md

# TEST-STRATEGY-{slug}
- Archetype + chosen pyramid ratio (+ justification for any deviation)
- Equivalence classes & boundaries enumerated per changed unit
- Property-based candidates (which logic, which invariants)
- Mutation-score target + result (for logic-dense changes)
- Quarantine list (flaky tests + their tracking bug)

Done = the plan answers "why THIS set of tests", not just "how much is covered".

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

ログインしてレビューを書く

同じリポジトリのスキル

概要と使いどころ

aesthetic-instrument

無料

great_cto's own committed aesthetic — the instrument panel. Dark five-step surface ladder, exactly one accent, two faces divided by MEANING (Geist speaks, Geist Mono is machine-truth), tabular numerals, and a dash that is not a nought. Twelve checkable rules measured from packages/board and greatcto.systems, not designed for this file. Use when building or extending any great_cto surface — the board, the site, a report, a share page.

日本語の概要は準備中です。原文の説明を表示しています。