risk-matrix
Overview
Per rbt-wiki:
"Risk-based testing (RBT) is 'a type of software testing that
functions as an organizational principle used to prioritize the
tests of features and functions in software, based on the risk
of failure.'"
The risk matrix is the artifact that drives RBT decisions. Without
it, prioritization is gut-feel; with it, the team has a
defensible record of "why we tested this and not that."
When to use
- A new feature is in planning; the team needs to scope test effort.
- A release scope is set; the team needs to allocate manual + auto
test time.
- Quarterly: the team wants to know which areas have the highest
unmitigated risk.
- Audit / compliance: the team must show a documented risk
assessment.
How to use
- Pick the methodology (Step 1) - lightweight impact-by-likelihood for most teams; heavyweight FMEA / Cost of Exposure only when regulation demands it.
- Intake each risk (Step 2) - one row per risk with feature, category, impact 1-5, likelihood 1-5, and the resulting score.
- Plot the heatmap and tier the verdict (Step 2) - block / mitigate-this-sprint / accept, using the team's block threshold (typically score 15+).
- Tag categories (Step 3) - business, technical, regulatory, UX, security, performance, integration - so patterns surface over time.
- Read the test plan off the matrix (Step 4) - map each top risk to its recommended test types.
- Set the cadence and store it in git (Steps 6-7) - the matrix is a living document reviewed per feature, per release, and quarterly.
- Map risks to coverage (Step 8) - before sign-off, show which tests, cases, or monitors back each risk and which risks are orphans.
The worked example below runs a checkout release through the full flow.
For the adjacent risk artifacts and workflows, see the
routing table at the end.
Step 1 - Pick the methodology
Per rbt-wiki, two approaches:
| Approach | Method | When |
|---|
| Lightweight | Impact × Likelihood, simple high/medium/low | Default; most teams. |
| Heavyweight | FMEA / Cost of Exposure / QFD / FTA | Regulated industries; safety-critical; insurance. |
Most teams should start lightweight - heavyweight methods need
specialized expertise.
Step 2 - Lightweight matrix structure
# Risk matrix - `<feature/release>`
**Date:** YYYY-MM-DD **Owner:** _______________ **Reviewers:** _______________
## Risks
| ID | Risk | Category | Impact (1-5) | Likelihood (1-5) | Score | Mitigation | Owner | Due |
|-----|-----------------------------------------------|------------|-------------:|-----------------:|------:|-------------------------------------|-------|-----|
| R-1 | Promo discount math wrong (off-by-cent) | Business | 5 | 3 | 15 | Add property-based tests on rounding | Alice | 2026-05-15 |
| R-2 | Stripe webhook delivery failure not retried | Technical | 4 | 4 | 16 | Add retry + DLQ; add chaos test | Bob | 2026-05-12 |
| R-3 | EU tax calculation incorrect | Regulatory | 5 | 2 | 10 | UAT with finance team | Carol | 2026-05-20 |
| R-4 | Cart loses state on app restart | UX | 3 | 3 | 9 | Persistent cart in localStorage | Dave | 2026-05-10 |
## Heatmap
| Likelihood ↓ \ Impact → | 1 (low) | 2 | 3 | 4 | 5 (high) |
|--------------------------|---------|---|---|---|----------|
| 5 (very likely) | | | | | |
| 4 | | | | R-2 | |
| 3 | | | R-4 | | R-1 |
| 2 | | | | | R-3 |
| 1 | | | | | |
## Verdict
- **Critical (score >=15):** R-1 (15), R-2 (16). Block release until mitigated.
- **High (score 9-14):** R-3 (10), R-4 (9). Mitigate this sprint.
- **Medium (score 5-8):** (none).
- **Low (score 1-4):** (none).
The 5×5 matrix yields scores 1-25; the team picks the threshold
for "block release" (typically 15+).
Step 3 - Risk categories
Per rbt-wiki, risks span:
- Business/operational: System criticality and usage frequency
- Technical: Team distribution and complexity
- External: Regulatory requirements and stakeholder preferences
- E-business specific: Security vulnerabilities, performance failures, and integration defects
Tag every risk with one category. Patterns emerge over time -
"all our top risks are integration" suggests an architectural
review, not just more testing.
Step 4 - Map risks to test types
A populated matrix drives the test plan:
| Risk class | Recommended test types |
|---|
| Business logic | Unit + property-based + UAT |
| Technical | Integration + chaos + load |
| Regulatory | UAT with stakeholder + compliance review |
| UX | Manual exploratory + visual regression |
| Security | Threat model + SAST + DAST + pen test |
| Performance | Load + perf budget + canary |
| Integration | Contract testing + integration tests + canary |
The test plan reads off the matrix: top-N risks → test types per
risk → estimated effort.
Worked example - checkout redesign release
Input: the checkout-redesign release scope, lightweight methodology (Step 1).
Intake and score (Steps 2-3). Four risks surface, each tagged with a
category and scored impact × likelihood:
| ID | Risk | Category | Impact | Likelihood | Score |
|---|
| R-1 | Promo discount math wrong (off-by-cent) | Business | 5 | 3 | 15 |
| R-2 | Stripe webhook delivery failure not retried | Technical | 4 | 4 | 16 |
| R-3 | EU tax calculation incorrect | Regulatory | 5 | 2 | 10 |
| R-4 | Cart loses state on app restart | UX | 3 | 3 | 9 |
Tier the verdict (Step 2). With a block threshold of 15: R-2 (16) and
R-1 (15) block the release until mitigated; R-3 (10) and R-4 (9) are high and
get mitigated this sprint.
Read the test plan off the matrix (Step 4). Each risk's category selects
its test types:
- R-1 (business logic) -> unit + property-based tests on rounding, plus UAT on the promo path.
- R-2 (technical / integration) -> retry + DLQ, a chaos test on webhook delivery, and contract tests against Stripe.
- R-3 (regulatory) -> UAT with the finance team plus a compliance review of the EU tax rules.
- R-4 (UX) -> manual exploratory plus visual-regression coverage of the persisted cart.
Outcome. The two blocking risks (R-1, R-2) get owners and due dates before
sign-off; the matrix is committed to
docs/risk-matrices/2026-Q2-checkout-redesign.md and re-reviewed at the next
cadence point (Step 6).
Step 5 - Heavyweight methods (FMEA, Cost of Exposure)
For regulated or safety-critical products where lightweight scoring is
insufficient, risk-based testing offers quantitative methods - FMEA (Risk
Priority Number = severity × occurrence × detectability) and Cost of Exposure
(annual financial risk vs mitigation cost) - per rbt-wiki. Both need
specialized expertise; use them only when regulation or financial
justification requires it. Full row structure, worked FMEA and
Cost-of-Exposure tables, and a when-to-use guide are in
references/heavyweight-risk-scoring.md.
Step 6 - Cadence
| Cadence | Trigger |
|---|
| Per-feature | Before development starts |
| Per-release | Pre-release sign-off |
| Quarterly | Strategic risk review |
| Post-incident | Update the matrix with the surfaced risk |
The matrix is a living document - risks change as features
ship, mitigations land, and incidents reveal new failure modes.
Step 7 - Format + storage
docs/risk-matrices/
├── 2026-Q2-checkout-redesign.md
├── 2026-Q2-stripe-integration.md
├── 2026-Q1-summary.md ← rollup
└── README.md
Markdown files version-controlled in git. Reviews via PR; updates
tracked over time.
Step 8 - Risk coverage mapping
A risk-coverage matrix proves every meaningful risk has a mitigation that
traces to a test or monitor - the risk-side complement to the requirements
traceability matrix (traceability-matrix-builder, in the qa-test-management
plugin). Per ISTQB CTAL-TM ch. 5 on risk-based test prioritisation and
ISO/IEC/IEEE 29119-3:2021 §6.3 on traceability (cite by stable ID). Run it
before a release sign-off or compliance audit, at sprint retrospectives to
find coverage debt, and in CI per PR.
8a - Ingest the risk register
Pull risks from the release matrix (this skill) and the product register
(references/product-risk-register.md);
project risks (schedule, staffing) are typically excluded - only product /
release risks map to test coverage. Filter to score >= 5 (Medium+).
8b - Inventory coverage from three sources
Tag tests with a risk:<ID> marker in name / docstring / front-matter, then:
grep -r "risk:R-001\|risk:PR-001" tests/ --include="*.py" \
--include="*.js" --include="*.ts" --include="*.java" -l
| Coverage source | How to collect |
|---|
| Automated tests | Repo scan for risk:<ID> tags (above) |
| Manual test cases | Query the TCM for cases whose refs field contains the risk ID |
| Production monitoring | A risk-coverage.yaml map of risk ID -> monitor IDs (e.g., datadog-monitor://stripe-webhook-failure-rate) |
8c - Compute depth and find orphans
Per risk, coverage depth = linked automated tests + manual cases + monitors:
| Depth | Verdict |
|---|
| 0 | Orphan risk - no coverage. Critical if risk score ≥ 10. |
| 1 | Minimal. Acceptable for low-score risks; insufficient for critical. |
| 2-4 | Reasonable. Multiple angles (unit + integration + monitor). |
| 5+ | Possibly over-tested. Audit for redundancy. |
Also run the reverse pass: tests whose risk:<ID> tag points at a retired /
non-existent risk are orphan tests - written for risks that no longer
exist. Audit for deletion.
8d - Emit the matrix + executive summary
Emit a Markdown document ordered by score descending: a header (total risks,
covered count and %, orphan count split critical vs low, average depth), one
row per risk (ID, title, score, automated tests, manual cases, monitors,
depth - bold any orphan), an "orphan risks (critical action)" section with a
recommended action + owner + estimate per orphan, an "over-covered (audit)"
list, and a coverage-debt trend table (orphan count and average depth over
recent months). Keep both artifacts version-controlled next to the matrix so
the trend is real history.
8e - CI gate
Re-build the matrix on every PR; fail if a critical-score risk becomes
uncovered:
- name: Risk coverage check
run: |
python scripts/build-risk-coverage.py \
--risks risks.yaml \
--output risk-coverage.md \
--fail-on-orphan-score 15
Caveats: tag discipline is the ceiling (untagged tests under-report
coverage); depth measures count, not test quality; a monitor existing does
not prove it would catch the risk; treat depth 1 as "covered" only for
low-score risks.
Going deeper (referenced guides)
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|
| Risk matrix authored once, never updated | Becomes irrelevant; team stops trusting. | Per-feature + quarterly cadence (Step 6). |
| Subjective scoring without examples | Different reviewers score differently; matrix unreliable. | Document score rubric (e.g., "5 = customer money loss"). |
| All-business-category matrix | Misses technical / regulatory risks; gaps invisible. | Tag every risk with a category (Step 3). |
| Matrix without owner per row | "Mitigated" never happens. | Owner column required (Step 2). |
| Heavyweight FMEA on a small product | Over-engineering; team disables. | Lightweight default; FMEA only when regulation requires (Step 1). |
| Risk matrix in slides, not version-controlled | History lost; rollup impossible. | Markdown + git (Step 7). |
Limitations
- Subjective scoring. Even with a rubric, team members weigh
risks differently. Multiple-reviewer consensus helps.
- Doesn't predict unknown unknowns. RBT addresses identified
risks; new failure modes emerge from incidents (and update the
matrix).
- Heavyweight methods need expertise. FMEA is well-defined; QFD
/ FTA require specialized training.
References
- rbt - Risk-based testing definition, lightweight vs
heavyweight methods (FMEA, Cost of Exposure, QFD, FTA), risk
categories (business / technical / external / e-business).
- ISTQB Advanced Test Manager (CTAL-TM) syllabus, ch. 5 - risk-based
testing + risk-coverage measurement (Step 8).
- ISTQB Glossary - glossary.istqb.org -
"coverage item", "test condition".
- ISO/IEC/IEEE 29119-3:2021 §6.3 - traceability (cite by stable ID).
test-strategy-author -
test strategy doc that references the matrix.traceability-matrix-builder (qa-test-management) - the
requirements-side complement of the Step 8 coverage matrix.