본문으로 건너뛰기
cccskills
무료GitHub에 공개

a-b-test-design

Plan A/B experiments to measure the effect of a design change. Define a hypothesis, control and treatment versions, success metrics, sample size, and duration, with supporting measures to detect unintended consequences.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

원문원문 설명 보기

Design an A/B experiment — hypothesis, variants, primary metric, and sample size. Use when a change can be measured quantitatively at scale. For observing behaviour qualitatively, use `test-scenario`.

설치 방법 보기

이럴 때 유용합니다

  • Planning a comparison between current and proposed designs
  • Defining success metrics for a change
  • Estimating sample size and test duration
  • Reviewing an experiment plan before launch

포함된 파일(1)

  • SKILL.md1.8 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Assess an existing interface against WCAG 2.2 accessibility guidelines. Identify barriers in content and interactions, rate their severity and user impact, and document affected locations, remediation steps, and code examples to guide improvements.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Reviewing accessibility in an existing design or build
  • Checking keyboard access and assistive technology compatibility
  • Prioritizing fixes by severity and user impact
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Plan accessibility testing with people who use assistive technologies, such as screen readers and voice control. Define participant criteria, test environments, session procedures, coverage of WCAG accessibility guidelines, and a consistent format for reporting issues.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Planning test sessions with assistive technology users
  • Defining participant recruitment criteria
  • Mapping key user flows to test environments
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Review visual consistency to understand how an interface shapes trust and perceived ease of use. Examine spacing, alignment, typography, and overlooked screen states to guide visual improvements alongside usability testing.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Investigating poor feedback on a functional interface
  • Checking visual consistency before launch
  • Reviewing error, empty, and loading states
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Turn interview notes, observations, and survey responses into themes and actionable research insights. Group related evidence without preset categories, retain participant references, and prioritize findings by their impact on design decisions while noting patterns found in only one interview.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Synthesize findings across multiple interviews
  • Organize observation notes and survey responses
  • Prioritize research insights for design decisions
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Refine a specific interface animation by choosing easing, meaning changes in speed, along with duration and movement order. Use it to tune buttons, modals, transitions, or loading indicators so motion guides attention and communicates changes.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Tune how a modal enters and exits
  • Adjust button and toggle animations
  • Coordinate the order of related movements
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Read conversion steps, retention curves, and recorded user actions to investigate why people leave a product. Check tracking issues first, then produce ranked design hypotheses with clear evidence that would support or rule out each explanation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Investigating where users abandon a signup flow
  • Interpreting changes in user retention
  • Checking tracking issues before changing a design
Owl-Listener/designer-skills3천2026년 9월 6일 업데이트

Owl-Listener의 스킬 모두 보기

이 스킬의 문제 신고