Design statistically rigorous A/B tests for product features, UI changes, onboarding flows, and pricing experiments. Use when asked to set up an experiment, design an A/B test, calculate sample size, or interpret test results. Produces a complete test plan with hypothesis, variant definitions, sample size, duration estimate, guardrail metrics, and a results interpretation guide.
インストールする前に、エージェントに与えられる指示の中身を確認できます。
Design experiments that produce trustworthy results — not just directional signals. Every test output includes hypothesis, success metrics, sample size, duration, and a results interpretation guide.
Work from what is given. If any of these is missing, still produce the complete output: fill each gap with a reasonable assumption marked (assumed, confirm) and list those assumptions at the end. Ask a question first only when the core subject itself is missing (there is nothing to write about).
Before running any test, confirm:
"We believe that [change] will cause [primary metric] to [increase/decrease] by [X%] for [user segment], because [rationale based on data or insight]."
Never run a test without a directional hypothesis. "Let's just see what happens" is not a hypothesis.
Use this formula (provide the output, not the formula, to the user):
For common scenarios, provide pre-calculated estimates:
| Baseline Rate | MDE (Relative) | Required Sample per Variant |
|---|---|---|
| 5% | 20% | ~19,000 |
| 10% | 15% | ~14,000 |
| 20% | 10% | ~15,000 |
| 40% | 10% | ~9,500 |
| 60% | 5% | ~42,000 |
Always warn: "These are estimates. Use a tool like Evan Miller's calculator or Statsig for precision."
Minimum: 2 full weeks (to capture weekly seasonality) Maximum: 4 weeks (novelty effect distorts results beyond this)
Duration = Required sample ÷ (Daily traffic × % exposed)
Flag if traffic is too low to reach significance in under 8 weeks — recommend a different approach (e.g., holdout test, qualitative research).
Hypothesis:
[Filled hypothesis template]
Variants:
Primary Metric: [Metric name + how measured] Guardrail Metrics: [Metrics that must not degrade]
Target Segment: [Who sees the test — % of traffic, user type] Traffic Split: [50/50 recommended unless ramp-up needed]
Sample Size Required: ~[N] users per variant Estimated Duration: [X] weeks (based on [Y] daily eligible users) Significance Threshold: 95% confidence, 80% power
Exclusions: [Any user segments to exclude and why]
Rollback Trigger: If [guardrail metric] degrades by [X%], stop the test immediately.
Results Interpretation Guide:
This skill ships with support files — use them when they are available:
references/test-validity-traps.md — The Validity Traps That Quietly Invalidate A/B Tests. Apply it while producing the output; it carries the calibration and judgment calls the method summary above compresses.templates/test-plan.md — a fill-in version of the deliverable with the quality gates inline. Offer it when the user wants to work the document themselves rather than have it generated.Score any output of this skill before handing it over; 32+ is ship-quality.
| Dimension | 0 | 5 | 10 |
|---|---|---|---|
| Statistical rigour | No sample size, or a number with no stated baseline/MDE behind it | Sample size present but MDE is guessed or copied from the lookup table without checking the actual baseline; power/significance unstated | Sample size derived from the stated baseline and MDE at 80% power / 95% confidence, duration checked against real daily traffic and the 2–4 week window, and the low-traffic escape hatch invoked if it doesn't fit |
| Hypothesis discipline | "Let's see what happens" — no direction, no magnitude, or multiple changes bundled into one variant | Directional hypothesis but missing magnitude, segment, or the evidence-based because; variant purity not confirmed | Full template filled (change, metric, direction, magnitude, segment, rationale citing data), and the treatment isolates exactly one change with excluded ideas named as follow-up tests |
| Guardrails & rollback | No guardrail metrics, or a rollback line with no threshold | Guardrails named but denominators/definitions ambiguous; rollback trigger vague ("if things look bad") | 1–2 guardrails protecting revenue or core engagement with pre-agreed definitions, concrete rollback thresholds, and the peeking-vs-harm-monitoring distinction handled explicitly |
| Decision readiness | No interpretation guide; results will be argued about after the fact | Ship/iterate/reject listed but thresholds fuzzy; inconclusive outcome missing or treated as a soft win | All four outcomes (ship / iterate / reject / inconclusive) mapped to pre-committed thresholds, including what an inconclusive result costs and what each outcome changes next |
まだレビューはありません。使ってみた感想をお寄せください。
概要と使いどころ
Design a 360-degree feedback survey or write a structured 360 feedback report. Use when asked to build a 360 feedback process, write 360 feedback for a colleague, design a feedback survey, or produce a feedback report. Produces either a complete survey instrument with rating scales and open-ended questions, or a structured narrative feedback report with themes, strengths, and development areas.
日本語の概要は準備中です。原文の説明を表示しています。
Decode a 401k or workplace retirement plan — the real cost of its funds, the match's fine print, vesting math, and the plan features worth using or avoiding. Use when someone asks 'is my 401k any good', 'decode my 401k plan', 'which funds should I look at', or 'what fees am I paying'. Produces a fee decode in dollars-over-time, match and vesting math, a fund-lineup triage by cost, and the questions for HR or the plan administrator.
日本語の概要は準備中です。原文の説明を表示しています。
Analyse a finished A/B test and write the readout — the result, whether it's statistically and practically significant, what it means, and the ship/no-ship call. Use when asked to analyse experiment results, write an A/B test readout, interpret test data, or decide whether to ship a variant. Produces a clear verdict with the lift and confidence, segment cuts, the risks (peeking, novelty, sample), and a recommendation. Distinct from planning a test — this reads results.
日本語の概要は準備中です。原文の説明を表示しています。
Build or tailor an academic CV for faculty, postdoc, research or fellowship applications: publications, grants, teaching, supervision and service in the order the field expects, with consistent citations. Use when asked to write my academic CV, update my CV for a lectureship, tenure-track or postdoc application, format my publications list, or adapt my academic CV for an industry research role. Produces the full academic CV, a field-appropriate section order, a clean publications list in one citation style, and a two-page summary version when asked.
日本語の概要は準備中です。原文の説明を表示しています。
Generate a WCAG 2.2 accessibility audit checklist and remediation suggestions for any UI or design. Use when asked to audit for accessibility, check WCAG compliance, review a design for a11y issues, or create an accessibility remediation plan. Produces a prioritised checklist with pass/fail assessments and specific fixes.
日本語の概要は準備中です。原文の説明を表示しています。
Plan a trip that actually works with a disability or access need — confirm real accessibility (not just 'accessible' labels), book the assistance in advance, plan for equipment and medication, and build in the contingencies for when access breaks down. Use when someone says 'plan an accessible trip', 'travelling with a wheelchair/disability', 'book assistance for my flight', or 'will this hotel actually work for me'. Produces an access-verified itinerary, an assistance-booking checklist, an equipment/medication plan, and contingency scripts. Verify specifics with providers.
日本語の概要は準備中です。原文の説明を表示しています。