跳到正文
cccskills
免费在 GitHub 公开

ab-test-readout

Analyse a finished A/B test and write the readout — the result, whether it's statistically and practically significant, what it means, and the ship/no-ship call. Use when asked to analyse experiment results, write an A/B test readout, interpret test data, or decide whether to ship a variant. Produces a clear verdict with the lift and confidence, segment cuts, the risks (peeking, novelty, sample), and a recommendation. Distinct from planning a test — this reads results.

中文简介正在准备中。当前显示原文说明。

查看安装方法

包含的文件(1)

  • SKILL.md3.1 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

A/B Test Readout Skill

The hard part of an experiment is the readout: not "B won" but "is this real, is it big enough to matter, and should we ship?" This skill turns results into an honest decision — and flags the ways A/B results lie.

Working from a brief

Given results (even partial), write the full readout anyway. If significance isn't provided, reason about it from the numbers and flag what's needed to confirm. Mark assumed figures. Never declare a winner without addressing significance and sample.

Required Inputs

Ask for (if not already provided):

  • The hypothesis and the primary metric
  • Results — control vs variant: conversions/rate, sample size per arm, duration
  • Guardrail metrics (revenue, retention, latency, complaints) that mustn't regress
  • Pre-registered decision rule (what would count as a win) if one exists

Output Format

1. Verdict (one line)

Ship / Don't ship / Inconclusive — keep running — with the headline number.

2. The result

MetricControlVariantRelative liftSignificant?
Primaryp / CI
Guardrail(s)

State statistical significance (p-value / confidence interval) and practical significance (is the lift big enough to matter given the cost?).

3. Did it really win?

Address the ways A/B tests mislead:

  • Sample / power — was the test adequately powered, or under-sampled?
  • Peeking — was the call made early, inflating false positives?
  • Novelty / primacy — could the effect fade?
  • Segments — does the win hold across key segments, or is it driven by one?

4. Segment cuts

Where the effect is strong vs flat vs negative (new vs returning, platform, geography).

5. Recommendation & next step

Ship / iterate / re-run, plus what to monitor post-launch or what the follow-up test should isolate.

Quality Checks

  • Distinguishes statistical from practical significance
  • Checks guardrail metrics, not just the primary
  • Flags peeking, power, novelty, and segment-driven wins
  • Recommendation follows from the evidence, with a monitoring/next-test step
  • Doesn't declare a winner on an underpowered or peeked result

Anti-Patterns

  • "B won by 8%!" with no significance or sample size
  • Calling a result early (peeking) and shipping
  • Ignoring a guardrail regression because the primary went up
  • A statistically significant but practically meaningless lift treated as a win

Example Trigger Phrases

  • "Analyse experiment results."
  • "Write an A/B test readout."
  • "Interpret test data."
  • "Decide whether to ship a variant."

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Design a 360-degree feedback survey or write a structured 360 feedback report. Use when asked to build a 360 feedback process, write 360 feedback for a colleague, design a feedback survey, or produce a feedback report. Produces either a complete survey instrument with rating scales and open-ended questions, or a structured narrative feedback report with themes, strengths, and development areas.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

Decode a 401k or workplace retirement plan — the real cost of its funds, the match's fine print, vesting math, and the plan features worth using or avoiding. Use when someone asks 'is my 401k any good', 'decode my 401k plan', 'which funds should I look at', or 'what fees am I paying'. Produces a fee decode in dollars-over-time, match and vesting math, a fund-lineup triage by cost, and the questions for HR or the plan administrator.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

Design statistically rigorous A/B tests for product features, UI changes, onboarding flows, and pricing experiments. Use when asked to set up an experiment, design an A/B test, calculate sample size, or interpret test results. Produces a complete test plan with hypothesis, variant definitions, sample size, duration estimate, guardrail metrics, and a results interpretation guide.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

Build or tailor an academic CV for faculty, postdoc, research or fellowship applications: publications, grants, teaching, supervision and service in the order the field expects, with consistent citations. Use when asked to write my academic CV, update my CV for a lectureship, tenure-track or postdoc application, format my publications list, or adapt my academic CV for an industry research role. Produces the full academic CV, a field-appropriate section order, a clean publications list in one citation style, and a two-page summary version when asked.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

Generate a WCAG 2.2 accessibility audit checklist and remediation suggestions for any UI or design. Use when asked to audit for accessibility, check WCAG compliance, review a design for a11y issues, or create an accessibility remediation plan. Produces a prioritised checklist with pass/fail assessments and specific fixes.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

Plan a trip that actually works with a disability or access need — confirm real accessibility (not just 'accessible' labels), book the assistance in advance, plan for equipment and medication, and build in the contingencies for when access breaks down. Use when someone says 'plan an accessible trip', 'travelling with a wheelchair/disability', 'book assistance for my flight', or 'will this hotel actually work for me'. Produces an access-verified itinerary, an assistance-booking checklist, an equipment/medication plan, and contingency scripts. Verify specifics with providers.

中文简介正在准备中。当前显示原文说明。

mohitagw15856/pm-claude-skills14812026年10月23日 更新

查看 mohitagw15856 的全部技能

报告此技能的问题