QA Persona Testing

Turn an evidence-backed ICP or user segment into a behavioral specification, drive its scenarios through browser automation, and report traced defects and preference hypotheses.

This is persona simulation, not E2E test authoring. It finds friction, confusion, trust breaks, and abandonment; it does not replace assertion-based regression tests (qa-testing-playwright) or research with real users (software-ux-research).

Quick Reference

Task	Read or use	Outcome
Find/derive the ICP or persona group	## Phase 1 — DEFINE + sibling skills table	Evidence-backed persona candidates
Turn a persona into an executable spec	assets/persona-profile.md + references/persona-construction.md	Persona profile with behavioral traits and scenario pack
Run a browser test session as the persona	references/browser-execution.md	Filled assets/session-log.md per scenario
Keep the simulation honest (anti-sycophancy)	references/persona-construction.md	Persona that abandons, complains, and stays in character
Write the improvement report	assets/findings-report.md + references/reporting-and-validity.md	Ranked findings, prioritized fixes, validity caveats
Decide what simulation can and cannot claim	references/reporting-and-validity.md	Correctly-hedged conclusions

When to Use

• "Test this app like a {{first-time user / busy accountant / 60-year-old iPad user / our ICP}}"
• "Would our target customer actually get through onboarding?"
• "Run a synthetic user / persona-based walkthrough and tell me what to improve"
• Pre-launch UX sanity pass when real-user testing is not yet feasible
• Regression of a redesigned flow against the personas that matter commercially

When NOT to Use

Situation	Use instead
Writing assertion-based E2E/regression tests	qa-testing-playwright
Research with real users (interviews, usability tests, surveys)	software-ux-research
Heuristic/accessibility audit without persona framing	software-ui-ux-design, qa-testing-accessibility
Validating whether the ICP itself is right for the business	startup-idea-validation
Testing an LLM agent or bot (agent is the system under test)	qa-agent-testing
Load, performance, or security testing	qa-testing-performance, qa-security-testing

ASCII Flow

persona-testing request
  -> Phase 1 DEFINE: find ICP / segment (sibling skills + evidence) -> persona candidates
  -> Phase 2 SETUP:  persona profile (traits, budgets, scenario pack, anti-sycophancy contract)
  -> Phase 3 EXECUTE: browser session per scenario as the persona
       +-- Playwright MCP or Chrome DevTools MCP (preferred)
       +-- Playwright script fallback (headless CI)
       -> session log: step trace, think-aloud, friction events, screenshots
  -> Phase 4 REPORT: rank findings by severity, then reproducibility
       -> improvements / avoid-list / keep-list / validity caveats
  -> Hand off: severity>=3 -> qa-testing-playwright regression tests
              preference findings -> software-ux-research real-user validation

Workflow

Phase 1 — DEFINE the profile

Goal: an evidence-backed persona, not an invented stereotype.

1. Ask (or infer from context) what decision the test must inform: launch readiness, redesign check, conversion friction, accessibility fit.
2. Source the ICP with the smallest sufficient chain of sibling skills:

Evidence you need	Skill to invoke
Who the ICP is at all (segment, JTBD, willingness to pay)	startup-idea-validation, startup-gtm-strategy
What real users complain about in this category	research-review-mining, research-painpoint-scanner
Behavioral data from an existing product (funnels, drop-offs)	marketing-product-analytics
Persona craft, research method, bias control	software-ux-research
Product context: roadmap, target users already defined	product-management, project-specific skills

3. Pick 1-3 personas maximum per run. Prefer one primary ICP plus one edge persona (low tech fluency or accessibility needs) — more personas dilute session depth.
4. Record provenance and confidence in the persona profile. An assumed persona is allowed but must be labeled assumed and its findings hedged accordingly.

Phase 2 — SETUP the persona

1. Copy assets/persona-profile.md; fill every field. Behavioral traits (patience, tech fluency, reading style, trust posture, error reaction) are the levers that make the simulation diverge from generic QA — do not leave them at defaults.
2. Write the scenario pack: 3-7 tasks phrased in the persona's words, each with an entry point, an observable success criterion, and the persona-specific risk it probes.
3. Adopt the anti-sycophancy contract (in the profile): patience budgets, mandatory abandonment, in-character confusion, no tester knowledge. See references/persona-construction.md for calibration and persona-drift checks.
4. Configure the environment to match the persona: viewport/device emulation, locale, network throttling if the persona is mobile/low-bandwidth.

Phase 3 — EXECUTE browser sessions

1. One session = one persona x one scenario. Start each session from the persona's real entry point (marketing page, app store link, shared URL) — not from a deep link the persona would never have.
2. Drive the app with browser tooling per references/browser-execution.md:
   • Playwright MCP (browser_navigate, browser_snapshot, browser_find, browser_click, ...) — default.
   • Chrome DevTools MCP — when you also need console/network/performance evidence, device/network emulation, or isolated browser contexts per scenario.
   • Playwright script — fallback for CI or when no MCP browser is available.
3. At every step, log the dual-channel trace into assets/session-log.md: in-character think-aloud + out-of-character observation + evidence (screenshot, console error, latency).
4. Enforce the persona: act only on what is visible in the snapshot, respect patience budgets, abandon when the persona would. An agent that always completes every task is a broken simulation.
5. Capture friction events with Nielsen severity 0-4 as they happen.

Phase 4 — REPORT and improve

1. Aggregate session logs into assets/findings-report.md: rank confirmed mechanical defects by severity, then independent reproducibility; keep unconfirmed observations in a verification queue. Report persona repetition separately; simulated frequency does not estimate real-user prevalence.
2. Every finding must cite a logged step and, where possible, a persona quote and evidence artifact. No finding without a trace.
3. Separate three lists: improve (prioritized fixes with expected effect), avoid (changes that would hurt this ICP), keep (flows that passed — protect with regression tests).
4. Apply the validity rules in references/reporting-and-validity.md: evidenced and independently reproduced mechanical failures (broken flows, errors, dead ends) are confirmed defects; emotional/preference findings are hypotheses for real-user validation. Name the missing analytics or real-user study needed to estimate prevalence or conversion impact.
5. Hand off follow-ups: severity ≥3 → qa-testing-playwright regression tests; top preference hypotheses → software-ux-research; ICP doubts surfaced by testing → startup-idea-validation.

Known Traps

• Sycophantic persona: the LLM completes every task and praises the app. Counter: patience budgets and mandatory abandonment rules are in the profile's Anti-Sycophancy Contract — enforce them, and run the sycophancy audit before shipping the report.
• Persona drift: over a long session the agent tends to slide back into QA-engineer voice (not yet measured at which step count). Counter: re-read the profile at each scenario start; keep the dual-channel log, and flag any step where the persona voice used tester vocabulary.
• Tester knowledge leak: persona "finds" a page via URL guessing or dev shortcuts. Counter: navigation only via visible UI from the declared entry point.
• Over-claiming: presenting simulated emotions as user research. Counter: the report's Validity section is mandatory, and preference findings are labeled hypotheses.
• Persona flattening / stereotyping: demographic caricature instead of behavioral spec. Counter: traits must map to observable behaviors (see profile trait table), and provenance must name evidence.
• The fidelity trap: flawless persona adherence reads as success but correlates with caricature — models with the highest persona fidelity produce the most stereotyped populations (arXiv:2604.24698). Counter: check sessions for stereotyped behavior as well as drift; discount findings that reduce to "this demographic behaved as expected". See references/persona-construction.md.
• One giant session: testing all scenarios in one browser context bleeds state (auth, carts, cookies) between scenarios. Counter: fresh context per session.

Navigation

Resources:

• references/persona-construction.md — ICP-to-persona conversion, behavioral trait calibration, anti-sycophancy contract, drift checks
• references/browser-execution.md — Playwright MCP / Chrome DevTools MCP / script execution patterns, evidence capture, environment emulation
• references/reporting-and-validity.md — severity model, ranking, report assembly, what simulation can and cannot claim
• data/sources.json — primary sources (research + tooling docs) with verification dates

Assets:

• assets/persona-profile.md — executable persona specification template
• assets/session-log.md — dual-channel (in/out of character) session trace template
• assets/findings-report.md — final report template (improve / avoid / keep / validity)

Related skills:

• ../qa-testing-playwright/SKILL.md — converts findings into durable E2E regression tests
• ../software-ux-research/SKILL.md — real-user validation of simulated findings; persona research methods
• ../software-ui-ux-design/SKILL.md — heuristic/WCAG framing for design-level fixes
• startup-idea-validation — ICP discovery and validation upstream of testing
• research-review-mining — real-user pain evidence to ground persona traits
• ../qa-agent-testing/SKILL.md — when the system under test is itself an agent
• ../qa-testing-accessibility/SKILL.md — deep accessibility audit beyond persona-level signals

Verification Notes

• Browser tooling behavior (Playwright MCP tool names, Chrome DevTools MCP capabilities, emulation flags) drifts fast — verify against current official docs before prescribing exact tool calls.
• Research claims about synthetic-user validity (what LLM personas do and do not reproduce) must cite the specific study; do not generalize a single benchmark result.
• Known bugs, framework footguns, and version-specific workarounds must be verified against current primary sources before being treated as current fact.
• Never present simulated persona reactions as evidence from real users.

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.