跳到正文
cccskills
免费在 GitHub 公开

ab-test-stats

Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills星标 150(GitHub 150 + cccskills 0)

实测评分 未实测

查看安装方法

包含的文件(3)

  • SKILL.md4.1 KB
  • scripts/main.py8.1 KB
  • scripts/requirements.txt76 B

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

A/B Test Statistics Calculator

Calculate statistical significance for A/B tests - know when your results are real, not random chance.

When to Use This Skill

  • Test analysis - Determine if results are statistically significant
  • Sample planning - Calculate required sample size before testing
  • Duration estimation - Know how long to run experiments
  • Power analysis - Ensure tests can detect meaningful differences

What Claude Does vs What You Decide

Claude DoesYou Decide
Structures analysis frameworksMetric definitions
Identifies patterns in dataBusiness interpretation
Creates visualization templatesDashboard design
Suggests optimization areasAction priorities
Calculates statistical measuresDecision thresholds

Dependencies

pip install scipy numpy click

Commands

Check Significance

python scripts/main.py significance --control 1000,50 --variant 1000,65
python scripts/main.py significance --control 5000,250 --variant 5000,300 --confidence 0.99

Calculate Sample Size

python scripts/main.py sample-size --baseline 0.05 --mde 0.02
python scripts/main.py sample-size --baseline 0.10 --mde 0.01 --power 0.90

Estimate Duration

python scripts/main.py duration --traffic 1000 --baseline 0.05 --mde 0.02

Examples

Example 1: Analyze Test Results

# Control: 1000 visitors, 50 conversions (5%)
# Variant: 1000 visitors, 65 conversions (6.5%)
python scripts/main.py significance --control 1000,50 --variant 1000,65

# Output:
# A/B Test Results
# ─────────────────────────
# Control:  5.00% (50/1000)
# Variant:  6.50% (65/1000)
# Lift:     +30.0%
#
# Statistical Analysis
# ─────────────────────────
# p-value:      0.089
# Confidence:   91.1%
# Result:       NOT SIGNIFICANT (need 95%)
#
# Recommendation: Continue test for more data

Example 2: Plan Sample Size

# Baseline 5% conversion, want to detect 20% relative lift (1% absolute)
python scripts/main.py sample-size --baseline 0.05 --mde 0.01

# Output:
# Sample Size Calculator
# ──────────────────────────────
# Baseline conversion: 5.0%
# Minimum detectable effect: 1.0% (20% relative)
# Target conversion: 6.0%
#
# Required per variant: 3,842 visitors
# Total required: 7,684 visitors
#
# At 1000 daily visitors: ~8 days

Key Concepts

TermDefinition
p-valueProbability result is due to chance
Confidence1 - p-value (usually want 95%+)
PowerProbability of detecting real effect (usually 80%)
MDEMinimum Detectable Effect - smallest lift worth detecting
LiftRelative improvement (variant - control) / control

When Results Are Significant

p-valueConfidenceVerdict
< 0.01> 99%Highly Significant ✓
< 0.05> 95%Significant ✓
< 0.10> 90%Marginally Significant
≥ 0.10< 90%Not Significant ✗

Skill Boundaries

What This Skill Does Well

  • Structuring data analysis
  • Identifying patterns and trends
  • Creating visualization frameworks
  • Calculating statistical measures

What This Skill Cannot Do

  • Access your actual data
  • Replace statistical expertise
  • Make business decisions
  • Guarantee prediction accuracy

Related Skills

Skill Metadata

  • Mode: centaur
category: analytics
subcategory: statistics
dependencies: [scipy, numpy]
difficulty: intermediate
time_saved: 3+ hours/week

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Measure and optimize growth using the AARRR (Pirate Metrics) framework with stage-specific KPIs and funnel analysis

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

Assess customer account health using product usage, support sentiment, payment status, and relationship signals

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

Analyze paid advertising performance across channels and recommend budget reallocation to maximize ROAS and minimize CAC. Use when: planning quarterly ad budget allocation, diagnosing underperforming ad channels, deciding whether to scale spend on a channel, calculating marginal ROI across Google Ads, Meta, LinkedIn, or TikTok, rebalancing media mix after performance shifts, or setting up a test-and-scale framework for new channels.

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

Use when testing skills, commands, or agents for quality. Use after creating new skills, before deploying agents, or when debugging inconsistent agent behavior. Triggers on "evaluate", "test quality", "is this skill working", or QA of AI workflows.

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

Use when analyzing server logs to understand how AI crawlers (GPTBot, ClaudeBot, PerplexityBot) interact with your site. Use when optimizing content placement for LLM retrieval, diagnosing why AI search isn't citing your content, or auditing crawl patterns to find optimization gaps.

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

Créez des storyboards visuellement cohérents en utilisant la technique des 2x2 Grid Shots de PJ Ace, garantissant éclairage, personnages et décors uniformes entre les plans. Use when: **Après avoir finalisé un script vidéo** - Transformer le concept en visuels; **Besoin de cohérence visuelle** - Personnages et éclairage constants entre les plans; **Préparer des assets pour animation** - Frames prêtes pour Veo, Runway, Kling; **Présenter un storyboard client** - Visualisation avant production;...

中文简介正在准备中。当前显示原文说明。

guia-matthieu/clawfu-skills1502026年10月1日 更新

查看 guia-matthieu 的全部技能

报告此技能的问题