免费在 GitHub 公开

a-b-test-design

Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.

中文简介正在准备中。当前显示原文说明。

Infrasity-Labs/dev-gtm-claude-skills

查看安装方法

包含的文件（1）

SKILL.md1.7 KB

SKILL.md（原文）

安装前可以查看交给 AI 代理的指令内容。

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

Control (A): current design
Treatment (B): proposed change
Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

Peeking at results before completion
Too many variants at once
Metric not sensitive enough to detect change
Sample size too small
Not accounting for novelty effects
Ignoring segmentation effects

When Not to A/B Test

Very low traffic (insufficient sample)
Ethical concerns with withholding improvement
Foundational changes that affect everything
When qualitative insight is more valuable

Best Practices

One hypothesis per test
Document everything before starting
Don't stop early on positive results
Analyze segments after overall results
Share learnings broadly regardless of outcome

评价

还没有评价。欢迎分享你的使用感受。

无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

同一仓库的技能

ab-testing

免费

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.

中文简介正在准备中。当前显示原文说明。