跳到正文
cccskills
免费在 GitHub 公开

a-b-test-design

Plan A/B experiments to measure the effect of a design change. Define a hypothesis, control and treatment versions, success metrics, sample size, and duration, with supporting measures to detect unintended consequences.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

Design an A/B experiment — hypothesis, variants, primary metric, and sample size. Use when a change can be measured quantitatively at scale. For observing behaviour qualitatively, use `test-scenario`.

实测评分 未实测

查看安装方法

适用场景

  • Planning a comparison between current and proposed designs
  • Defining success metrics for a change
  • Estimating sample size and test duration
  • Reviewing an experiment plan before launch

包含的文件(1)

  • SKILL.md1.8 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Assess an existing interface against WCAG 2.2 accessibility guidelines. Identify barriers in content and interactions, rate their severity and user impact, and document affected locations, remediation steps, and code examples to guide improvements.

中文简介正在准备中。当前显示英语简介。

  • Reviewing accessibility in an existing design or build
  • Checking keyboard access and assistive technology compatibility
  • Prioritizing fixes by severity and user impact
Owl-Listener/designer-skills30552026年9月6日 更新

Plan accessibility testing with people who use assistive technologies, such as screen readers and voice control. Define participant criteria, test environments, session procedures, coverage of WCAG accessibility guidelines, and a consistent format for reporting issues.

中文简介正在准备中。当前显示英语简介。

  • Planning test sessions with assistive technology users
  • Defining participant recruitment criteria
  • Mapping key user flows to test environments
Owl-Listener/designer-skills30552026年9月6日 更新

Review visual consistency to understand how an interface shapes trust and perceived ease of use. Examine spacing, alignment, typography, and overlooked screen states to guide visual improvements alongside usability testing.

中文简介正在准备中。当前显示英语简介。

  • Investigating poor feedback on a functional interface
  • Checking visual consistency before launch
  • Reviewing error, empty, and loading states
Owl-Listener/designer-skills30552026年9月6日 更新

Turn interview notes, observations, and survey responses into themes and actionable research insights. Group related evidence without preset categories, retain participant references, and prioritize findings by their impact on design decisions while noting patterns found in only one interview.

中文简介正在准备中。当前显示英语简介。

  • Synthesize findings across multiple interviews
  • Organize observation notes and survey responses
  • Prioritize research insights for design decisions
Owl-Listener/designer-skills30552026年9月6日 更新

Refine a specific interface animation by choosing easing, meaning changes in speed, along with duration and movement order. Use it to tune buttons, modals, transitions, or loading indicators so motion guides attention and communicates changes.

中文简介正在准备中。当前显示英语简介。

  • Tune how a modal enters and exits
  • Adjust button and toggle animations
  • Coordinate the order of related movements
Owl-Listener/designer-skills30552026年9月6日 更新

Read conversion steps, retention curves, and recorded user actions to investigate why people leave a product. Check tracking issues first, then produce ranked design hypotheses with clear evidence that would support or rule out each explanation.

中文简介正在准备中。当前显示英语简介。

  • Investigating where users abandon a signup flow
  • Interpreting changes in user retention
  • Checking tracking issues before changing a design
Owl-Listener/designer-skills30552026年9月6日 更新

查看 Owl-Listener 的全部技能

报告此技能的问题