Skip to content
cccskills
FreePublished on GitHub

test-strategy

Coverage-design method for qa-engineer — pyramid ratios per archetype, equivalence/boundary/property case selection, mutation score as the real coverage signal, and a flake-quarantine policy. Turns "coverage is 90%" (a number with no method) into a defensible test plan. Emits TEST-STRATEGY-{slug}.md, which the QA gate checks exists.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto
See how to install

Files included (1)

  • SKILL.md4.0 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

test-strategy

Coverage % is a number with no method. This skill makes the QA plan defensible: what to test, why that set, and a real signal that the tests bite.

qa-engineer emits docs/qa-reports/TEST-STRATEGY-{slug}.md capturing the four decisions below; the QA gate checks the file exists for medium/large features.

1. Pyramid ratio — pick by archetype (not by reflex)

The right unit:integration:e2e mix depends on where the risk lives.

Archetypeunitintegratione2ewhy
library / devtools / cli80%15%5%logic-dense, few I/O seams
web-app / saas60%30%10%request→db→render seams dominate
commerce / fintech / marketplace50%35%15%money paths need cross-component proof
data-platform / streaming45%45%10%correctness lives in pipelines, not units
ai-system / agent-product50%20%10% + evals 20%behaviour is the contract → eval set (see [[decision-eval]])

State the chosen ratio in TEST-STRATEGY and justify any deviation.

2. Case selection — equivalence / boundary / property

Don't enumerate inputs; partition them.

  • Equivalence classes — one representative per class of behaviour (valid, invalid, empty, max). Testing 5 valid ids ≠ testing 5 classes.
  • Boundaries — the bug lives at the edge: 0, 1, n-1, n, n+1, off-by-one, empty, overflow, the threshold itself.
  • Property-based — for pure/transform logic, assert invariants over generated input (round-trip decode(encode(x))===x, idempotency, ordering) instead of hand-picked cases. Use fast-check / Hypothesis where the logic warrants it.

3. Mutation score — the only coverage that proves the tests bite

Line/branch coverage proves code ran, not that a test would fail if it broke. Mutation testing (Stryker / mutmut / cargo-mutants) flips operators/conditions and checks a test catches it. A "90% covered" module with 30% mutation score has assertion-free tests. Target: mutation score ≥ 60% on changed logic-dense files; report it in TEST-STRATEGY when the change is logic-heavy. Coverage % is the cheap proxy; mutation score is the truth.

4. Flake quarantine — a flaky test is worse than no test

A test that fails ~randomly trains the team to ignore red. Policy:

  • A test that fails non-deterministically is quarantined (skipped + tracked in a Beads bug), not left to rot the signal.
  • Quarantine is time-boxed: a bug with an owner and a deadline, not it.skip forever.
  • Root-cause before re-enabling (shared state, timing, real network, ordering).
  • This is the same discipline the eval runner needs (see the eval flaky flag in tests/eval/runner.mjs — stddev > 0.1 is a flake signal).

Output: TEST-STRATEGY-{slug}.md

# TEST-STRATEGY-{slug}
- Archetype + chosen pyramid ratio (+ justification for any deviation)
- Equivalence classes & boundaries enumerated per changed unit
- Property-based candidates (which logic, which invariants)
- Mutation-score target + result (for logic-dense changes)
- Quarantine list (flaky tests + their tracking bug)

Done = the plan answers "why THIS set of tests", not just "how much is covered".

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

aesthetic-instrument

Free

great_cto's own committed aesthetic — the instrument panel. Dark five-step surface ladder, exactly one accent, two faces divided by MEANING (Geist speaks, Geist Mono is machine-truth), tabular numerals, and a dash that is not a nought. Twelve checkable rules measured from packages/board and greatcto.systems, not designed for this file. Use when building or extending any great_cto surface — the board, the site, a report, a share page.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

anti-patterns

Free

Catalogue of known SDLC anti-patterns that great_cto agents must actively reject when reviewing architecture, plans, code, or post-mortems. Used by architect (pre-impl), pm (planning), senior-dev (impl), l3-support (post-incident).

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

anydesign

Free

Analyze images, websites, and Figma files to extract their design and generate a `design.md` with token system, component inventory, and reconstruction notes. Use this skill whenever the user wants to understand, document, replicate, or audit the design of something visual: a screenshot, a URL, a Figma link, a Pinterest reference, a mockup, a competitor's site, a component, a dashboard, a landing page. Also when they ask 'extract the design system from X', 'document the style of Y', 'analyze this visually', 'convert this image into tokens', 'help me replicate this design', 'what palette does this site use', 'how is this built'. Also for single elements: 'copy this navbar', 'recreate this illustration', 'give me a prompt to regenerate this graphic' — element mode outputs a focused element.md, with token-grounded image-model prompts when the element is visual art. If the user brings any visual source and wants to understand it at a design level — this skill should activate.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

archetype-review-base

Free

Shared review framework that every domain reviewer (pci, oracle, gov, edtech, healthcare, mlops, etc.) MUST follow. Defines the output artifact (TM-{slug}.md), mandatory sections, severity scale, verdict format, the workflow scaffold (when-invoked, Step-0 read-inputs, HANDOFF), and the "domain heuristic vs generic check" boundary. Eliminates duplication across the ~30 reviewer prompts.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

brainstorming

Free

Structured idea generation + multi-LLM debate for the product-owner stage. Diverge (generate genuinely different bets), debate (a 4-persona panel on 4 models argues over 2 rounds), converge (synthesize a recommendation). Used by product-owner before architect; available to architect for design-space exploration.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

codex-host

Free

Run the great_cto controlled Codex lifecycle with controller-owned writes, verifier evidence, human gates and optional artifact release.

The English summary is being prepared. Showing the original description.

avelikiy/great_cto105Updated Oct 23, 2026

See all skills by avelikiy

Report a problem with this skill