跳到正文
cccskills
免费在 GitHub 公开

experiment-designer

Plan and evaluate product A/B tests with clear hypotheses, success metrics, sample size estimates, and stopping rules. Rank proposed experiments and interpret results using effect sizes and uncertainty to support practical product decisions.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

Use when planning product experiments, writing testable hypotheses, estimating sample size, prioritizing tests, or interpreting A/B outcomes with practical statistical rigor.

查看安装方法

适用场景

  • Turn a product improvement idea into a testable hypothesis
  • Estimate the sample size needed for an A/B test
  • Rank experiments by impact, confidence, and ease
  • Interpret test results alongside business impact

包含的文件(4)

  • SKILL.md3.1 KB
  • references/experiment-playbook.md1.9 KB
  • references/statistics-reference.md1.5 KB
  • scripts/sample_size_calculator.py3.0 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Experiment Designer

Design, prioritize, and evaluate product experiments with clear hypotheses and defensible decisions.

When To Use

Use this skill for:

  • A/B and multivariate experiment planning
  • Hypothesis writing and success criteria definition
  • Sample size and minimum detectable effect planning
  • Experiment prioritization with ICE scoring
  • Reading statistical output for product decisions

Core Workflow

  1. Write hypothesis in If/Then/Because format
  • If we change [intervention]
  • Then [metric] will change by [expected direction/magnitude]
  • Because [behavioral mechanism]
  1. Define metrics before running test
  • Primary metric: single decision metric
  • Guardrail metrics: quality/risk protection
  • Secondary metrics: diagnostics only
  1. Estimate sample size
  • Baseline conversion or baseline mean
  • Minimum detectable effect (MDE)
  • Significance level (alpha) and power

Use:

python3 scripts/sample_size_calculator.py --baseline-rate 0.12 --mde 0.02 --mde-type absolute
  1. Prioritize experiments with ICE
  • Impact: potential upside
  • Confidence: evidence quality
  • Ease: cost/speed/complexity

ICE Score = (Impact * Confidence * Ease) / 10

  1. Launch with stopping rules
  • Decide fixed sample size or fixed duration in advance
  • Avoid repeated peeking without proper method
  • Monitor guardrails continuously
  1. Interpret results
  • Statistical significance is not business significance
  • Compare point estimate + confidence interval to decision threshold
  • Investigate novelty effects and segment heterogeneity

Hypothesis Quality Checklist

  • Contains explicit intervention and audience
  • Specifies measurable metric change
  • States plausible causal reason
  • Includes expected minimum effect
  • Defines failure condition

Common Experiment Pitfalls

  • Underpowered tests leading to false negatives
  • Running too many simultaneous changes without isolation
  • Changing targeting or implementation mid-test
  • Stopping early on random spikes
  • Ignoring sample ratio mismatch and instrumentation drift
  • Declaring success from p-value without effect-size context

Statistical Interpretation Guardrails

  • p-value < alpha indicates evidence against null, not guaranteed truth.
  • Confidence interval crossing zero/no-effect means uncertain directional claim.
  • Wide intervals imply low precision even when significant.
  • Use practical significance thresholds tied to business impact.

See:

  • references/experiment-playbook.md
  • references/statistics-reference.md

Tooling

scripts/sample_size_calculator.py

Computes required sample size (per variant and total) from:

  • baseline rate
  • MDE (absolute or relative)
  • significance level (alpha)
  • statistical power

Example:

python3 scripts/sample_size_calculator.py \
  --baseline-rate 0.10 \
  --mde 0.015 \
  --mde-type absolute \
  --alpha 0.05 \
  --power 0.8

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Audit web applications for accessibility issues against WCAG 2.2 Level A and AA, the web accessibility standards. Get framework-specific code fixes, contrast checks, verification steps, and reports, with options for automated checks in development pipelines.

中文简介正在准备中。当前显示英语简介。

  • Audit a web application before release
  • Fix contrast and keyboard navigation issues
  • Prepare accessibility reports for stakeholders
alirezarezvani/claude-skills2.9万2026年8月30日 更新

Plan A/B tests that compare page variations, from headlines to pricing layouts. Define a hypothesis, choose success metrics, calculate sample size and duration, and prepare launch checks and results reports to support decisions.

中文简介正在准备中。当前显示英语简介。

  • Comparing landing page headlines
  • Testing signup button changes
  • Planning pricing page experiments
alirezarezvani/claude-skills2.9万2026年8月30日 更新

Create paid ad headlines and body copy tailored to each platform's character limits. Use existing performance data to develop variations, explore new messaging angles, and check copy for length, quality, and potential rejection risks.

中文简介正在准备中。当前显示英语简介。

  • Creating Google RSA headlines
  • Writing Meta and LinkedIn ad copy
  • Developing variations of successful ads
alirezarezvani/claude-skills2.9万2026年8月30日 更新

Review code changes from three critical perspectives: production failures, maintainability, and security. Get a consolidated report that ranks findings by severity and provides merge guidance, helping examine assumptions that an ordinary self-review may overlook.

中文简介正在准备中。当前显示英语简介。

  • Reviewing a pull request before merging
  • Getting a second opinion on AI-written code
  • Checking authentication or data access code
alirezarezvani/claude-skills2.9万2026年8月30日 更新

aeo

免费

Audit and improve content intended for citation in AI answers. Review evidence, author information, and structure, generate revised content and structured metadata, and keep a local record of observed citations across AI services.

中文简介正在准备中。当前显示英语简介。

  • Audit existing articles for evidence and credibility gaps
  • Revise article structure and references for AI answers
  • Record citations observed across AI services
alirezarezvani/claude-skills2.9万2026年8月30日 更新

Create signed records for consequential agent actions such as deployments, deletions, and payments. Decide which actions need evidence, record the agent, action, target, and policy, then verify the resulting certificate offline to detect later changes.

中文简介正在准备中。当前显示英语简介。

  • Creating signed records for production deployments
  • Preserving evidence of deletions and payments
  • Recording access grants for later review
alirezarezvani/claude-skills2.9万2026年8月30日 更新

查看 alirezarezvani 的全部技能

报告此技能的问题