test-strategy-author
Overview
A test strategy document answers the question "how do we know
we're testing the right things?" Without it, test investment is
gut-feel; with it, the team has a defensible artifact tying test
work to risk and business goals.
This skill produces the document.
When to use
- A new product / large feature is in planning; the team needs to
scope test investment.
- A multi-quarter initiative needs a published strategy stakeholders
can review.
- An audit / compliance review requires a documented strategy.
- A new team forming around a product needs onboarding context.
For per-feature test plans (smaller scope), use the lighter-weight
risk-matrix - strategy is for
larger horizons.
Step 1 - Document structure
# Test Strategy - `<product / release>`
**Author:** _______________ **Date:** YYYY-MM-DD
**Status:** Draft | Review | Approved
**Stakeholders:** Engineering, Product, QA, Compliance
## 1. Scope
### In scope
- (List of features / surfaces this strategy covers)
### Out of scope
- (Explicit list of what's NOT covered, with rationale)
### Assumptions
- (List of context the strategy depends on)
## 2. Risk basis
(Reference the risk matrix per `risk-matrix`)
| Risk class | Top risks (from matrix) | Test investment |
|------------|-------------------------|-----------------|
| Business | Promo math, Tax calculations | Property-based + UAT |
| Technical | Webhook reliability, DB migrations | Chaos + integration |
| Regulatory | EU GDPR, CCPA | UAT + privacy review |
| Performance | Checkout latency | Load + canary |
## 3. Test types per layer (the pyramid)
Per the test pyramid (`test-pyramid-balancer`):
| Layer | Coverage target | Tools | Owner |
|-------------|----------------:|-------|-------|
| Unit | 80% | Jest, pytest, JUnit | Devs |
| Integration | 60% | Testcontainers, supertest | Devs |
| Contract | 100% of consumer-provider pairs | Pact | Devs |
| E2E | Critical paths (5-10 flows) | Playwright | QA |
| Performance | Critical endpoints | k6 + Lighthouse CI | QA + SRE |
| Security | OWASP Top 10 | Schemathesis + manual pen test | Security |
| A11y | WCAG 2.2 AA | axe + manual review | QA |
## 4. Tooling
(Stack inventory per layer; references to specific skills)
## 5. Environments
- **Local dev** - per-engineer; Testcontainers backing services.
- **Staging** - shared; smoke + UAT.
- **Canary** - 5% prod traffic; 30-min observation per
prod-canary-validator.
- **Prod** - synthetic monitors per
synthetic-monitor-author.
## 6. Test data
- **Synthetic accounts** - per synthetic-data-toolkit.
- **PII handling** - per synthetic-pii-generator.
- **Database state** - per database snapshot / restore.
## 7. Exit criteria
Release ships when:
- All AC for in-scope features pass.
- All unit/integration tests green; no flake in last 3 main runs.
- E2E critical-path suite green.
- Coverage targets met per Section 3.
- Performance targets met (p95 within budget).
- A11y regression scan green.
- Threat model reviewed for security-touching changes.
- Risk matrix Critical (>=15) all mitigated.
## 8. Ownership
| Activity | Owner | Backup |
|-----------------------|-------|--------|
| Unit test reviews | Devs | Tech lead |
| E2E suite maintenance | QA | Dev TPM |
| Perf budget approval | SRE | QA |
| Threat model authorship | Security | Dev TPM |
| Synthetic monitors | SRE | QA |
| Risk matrix updates | QA | Product |
## 9. Cadence
- **Per-PR:** Lint, unit, integration, smoke E2E, coverage delta.
- **Per-merge to main:** Full E2E, perf gate.
- **Nightly:** Full regression, mutation testing weekly.
- **Pre-release:** Manual UAT sign-off, full security scan,
release-readiness check.
## 10. Risk register snapshot
(Top 10 risks from the matrix as of strategy authoring date)
## 11. Open questions / decisions needed
- (List of unresolved items requiring stakeholder input)
## Approval
- [ ] Engineering manager
- [ ] QA lead
- [ ] Product manager
- [ ] Security (if applicable)
Step 1a - Validate the draft before approval
Before circulating for sign-off, verify the draft against its own exit gate:
- Assert every in-scope feature in Section 1 maps to a risk class in Section 2, a layer + coverage target in Section 3, and at least one exit criterion in Section 7.
- Assert every Section 7 exit criterion is measurable (a number or a green/red check), not a subjective phrase.
If a section cannot be filled - for example the risk matrix in Section 2 does not exist yet - record it as an open question in Section 11 rather than leaving the section blank, resolve it, then re-run this check. Collect the Approval sign-offs only once no Section 11 item still blocks release.
Step 2 - Tailor per project size
| Project size | Strategy length | Scope |
|---|
| Small (1-3 mo) | 2-3 pages | Sections 1-3 + 7 |
| Medium (1-2 quarters) | 5-7 pages | All sections |
| Large (multi-quarter) | 10+ pages | All + sub-strategies per major feature |
Step 3 - Review cadence
| Trigger | Action |
|---|
| Quarterly | Re-review; update sections 2 (risks), 7 (exit), 10 (snapshot). |
| Major architectural change | Re-author Sections 3 + 6. |
| Post-incident | Update Section 2 (new risks); Section 7 (new exit criterion). |
| New compliance requirement | Section 1 + Section 7. |
Step 4 - Reference patterns
Common strategy patterns by product type:
| Product type | Strategy emphasis |
|---|
| E-commerce | Business risks (promo math, currency, tax); UAT-heavy. |
| SaaS B2B | Security + multi-tenancy; contract testing critical. |
| Mobile native | Per-platform matrix; manual tier; performance. |
| Internal tooling | Light strategy; integration over E2E. |
| Regulated (fintech, health) | Heavyweight RBT (FMEA); auditable trail. |
Step 5 - Risk-based test planning (per feature / per quarter)
The strategy sets the frame; a per-feature plan allocates the hours. Given a
feature scope, the current risk matrix (per risk-matrix), an engineer-week
budget, and the team's tooling inventory, produce a risk-prioritized test
plan:
- Identify scope-relevant risks. Direct hits (the feature modifies code
in a risk's source paths) plus area-level hits (same product area as an
existing risk row). No matrix, no plan - author the matrix first.
- Map each risk to test types per
risk-matrix Step 4 (business logic
-> unit + property-based + UAT; technical -> integration + chaos + load;
regulatory -> stakeholder UAT + compliance review; and so on).
- Estimate effort per test type (rough starting points the team
calibrates): unit 0.5h, property-based 2h, integration / contract 4h,
E2E 8h, manual / UAT 4h per scenario, visual regression 2h per baseline,
load 1d, chaos 2d, threat model 1d.
- Emit the plan with a per-risk table (risk, score, class, test types,
effort, owner), a budget-allocation summary with ~20% headroom for
incident response and retests, and a "Risks NOT addressed
(intentional)" table naming each skipped risk and why - the audit trail
that distinguishes prioritization from neglect.
## Test plan - Feature `Promo banner v2`
**Risks implicated:** 6 (of 23) · **Budget:** 2 engineer-weeks · **Estimated:** 1.6 weeks (within budget)
| Risk | Score | Class | Test types | Effort | Owner |
|---|---:|---|---|---|---|
| R-1 Promo math | 15 | Business logic | + 4 unit + 1 property-based | 4 hours | Alice |
| R-2 Stripe webhook | 16 | Technical | + 1 integration + 1 chaos | 1.5 days | Bob |
| R-3 EU tax calc | 10 | Regulatory | + 1 UAT with finance | 4 hours | Carol |
### Risks NOT addressed (intentional)
| Risk | Score | Why skipped |
|---|---:|---|
| R-15 Old promo CMS migration | 3 | Below threshold; manual smoke covers. |
Planning guardrails: never plan more than 20% over budget; never skip the
highest-scored Critical risk (>=15) under budget pressure - escalate to
product instead; never pick test types the team has no tooling for. Version
plans in markdown alongside the matrix and track revisions rather than
silently rewriting prior versions.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|
| Generic strategy template not tailored to product | Reads like boilerplate; nobody references. | Tailor per Section 1 (scope) and Section 4 (tooling). |
| Plan ignores the effort budget | Idealistic; team can't execute. | Match plan to budget (Step 5). |
| Skipping Critical risks under budget pressure | Defeats the prioritization purpose. | Escalate to product instead (Step 5 guardrails). |
| No "risks NOT addressed" section in the plan | Audit gap; prioritization indistinguishable from neglect. | Always include it (Step 5). |
| Strategy authored once; never reviewed | Becomes obsolete; team distrusts. | Quarterly cadence (Step 3). |
| No exit criteria | "Done" is subjective; release decisions arbitrary. | Section 7 - explicit + measurable. |
| Strategy = test plan | Strategy is high-level; per-feature plans are detailed (different artifact). | Use risk-matrix for per-feature. |
| Owner column missing | Activities slip; "everyone owns it = nobody." | Section 8 explicit owners. |
| Strategy in slides, not version-controlled | History lost; updates invisible. | Markdown + git. |
Limitations
- Strategy ≠ execution. A great strategy doesn't ensure great
testing - pair with per-feature plans + per-PR enforcement.
- Quarterly cadence is a floor. Fast-moving products may need
more frequent revisits.
- Stakeholder buy-in required. A strategy nobody read is
paperwork; circulate, present, get sign-off.
References
risk-matrix - feeds Section 2.test-pyramid-balancer -
feeds Section 3.definition-of-done - feeds
Section 7.