Skip to content
cccskills
FreePublished on GitHub

a-b-test-design

Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills

Run score Not tested

See how to install

Files included (1)

  • SKILL.md1.7 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

ab-testing

Free

When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

academic-cv-builder

Free

Format CVs for academic positions with publications, grants, and teaching

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

accessibility-audit

Free

Conduct a comprehensive accessibility audit against WCAG guidelines with severity ratings and remediation steps.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

accessibility-test-plan

Free

Create accessibility testing plans covering assistive technologies and WCAG criteria.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

ad-creative

Free

When the user wants to generate, iterate, or scale ad creative — headlines, descriptions, primary text, or full ad variations — for any paid advertising platform. Also use when the user mentions 'ad copy variations,' 'ad creative,' 'generate headlines,' 'RSA headlines,' 'bulk ad copy,' 'ad iterations,' 'creative testing,' 'ad performance optimization,' 'write me some ads,' 'Facebook ad copy,' 'Google ad headlines,' 'LinkedIn ad text,' or 'I need more ad variations.' Use this whenever someone needs to produce ad copy at scale or iterate on existing ads. For campaign strategy and targeting, see ads. For landing page copy, see copywriting.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

ads

Free

When the user wants help with paid advertising campaigns on Google Ads, Meta (Facebook/Instagram), LinkedIn, Twitter/X, or other ad platforms. Also use when the user mentions 'PPC,' 'paid media,' 'ROAS,' 'CPA,' 'ad campaign,' 'retargeting,' 'audience targeting,' 'Google Ads,' 'Facebook ads,' 'LinkedIn ads,' 'ad budget,' 'cost per click,' 'ad spend,' or 'should I run ads.' Use this for campaign strategy, audience targeting, bidding, and optimization. For bulk ad creative generation and iteration, see ad-creative. For landing page optimization, see cro.

The English summary is being prepared. Showing the original description.

Infrasity-Labs/dev-gtm-claude-skills143Updated Jun 29, 2026

See all skills by Infrasity-Labs

Report a problem with this skill