Calculate A/B test statistical significance. Use when: determining if test results are significant; calculating required sample size; estimating test duration; analyzing conversion experiments; making data-driven decisions
The English summary is being prepared. Showing the original description.
Run score Not tested
See how to installReview the instructions the agent will receive before you install.
Calculate statistical significance for A/B tests - know when your results are real, not random chance.
| Claude Does | You Decide |
|---|---|
| Structures analysis frameworks | Metric definitions |
| Identifies patterns in data | Business interpretation |
| Creates visualization templates | Dashboard design |
| Suggests optimization areas | Action priorities |
| Calculates statistical measures | Decision thresholds |
pip install scipy numpy click
python scripts/main.py significance --control 1000,50 --variant 1000,65
python scripts/main.py significance --control 5000,250 --variant 5000,300 --confidence 0.99
python scripts/main.py sample-size --baseline 0.05 --mde 0.02
python scripts/main.py sample-size --baseline 0.10 --mde 0.01 --power 0.90
python scripts/main.py duration --traffic 1000 --baseline 0.05 --mde 0.02
# Control: 1000 visitors, 50 conversions (5%)
# Variant: 1000 visitors, 65 conversions (6.5%)
python scripts/main.py significance --control 1000,50 --variant 1000,65
# Output:
# A/B Test Results
# ─────────────────────────
# Control: 5.00% (50/1000)
# Variant: 6.50% (65/1000)
# Lift: +30.0%
#
# Statistical Analysis
# ─────────────────────────
# p-value: 0.089
# Confidence: 91.1%
# Result: NOT SIGNIFICANT (need 95%)
#
# Recommendation: Continue test for more data
# Baseline 5% conversion, want to detect 20% relative lift (1% absolute)
python scripts/main.py sample-size --baseline 0.05 --mde 0.01
# Output:
# Sample Size Calculator
# ──────────────────────────────
# Baseline conversion: 5.0%
# Minimum detectable effect: 1.0% (20% relative)
# Target conversion: 6.0%
#
# Required per variant: 3,842 visitors
# Total required: 7,684 visitors
#
# At 1000 daily visitors: ~8 days
| Term | Definition |
|---|---|
| p-value | Probability result is due to chance |
| Confidence | 1 - p-value (usually want 95%+) |
| Power | Probability of detecting real effect (usually 80%) |
| MDE | Minimum Detectable Effect - smallest lift worth detecting |
| Lift | Relative improvement (variant - control) / control |
| p-value | Confidence | Verdict |
|---|---|---|
| < 0.01 | > 99% | Highly Significant ✓ |
| < 0.05 | > 95% | Significant ✓ |
| < 0.10 | > 90% | Marginally Significant |
| ≥ 0.10 | < 90% | Not Significant ✗ |
category: analytics
subcategory: statistics
dependencies: [scipy, numpy]
difficulty: intermediate
time_saved: 3+ hours/week
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Measure and optimize growth using the AARRR (Pirate Metrics) framework with stage-specific KPIs and funnel analysis
The English summary is being prepared. Showing the original description.
Assess customer account health using product usage, support sentiment, payment status, and relationship signals
The English summary is being prepared. Showing the original description.
Analyze paid advertising performance across channels and recommend budget reallocation to maximize ROAS and minimize CAC. Use when: planning quarterly ad budget allocation, diagnosing underperforming ad channels, deciding whether to scale spend on a channel, calculating marginal ROI across Google Ads, Meta, LinkedIn, or TikTok, rebalancing media mix after performance shifts, or setting up a test-and-scale framework for new channels.
The English summary is being prepared. Showing the original description.
Use when testing skills, commands, or agents for quality. Use after creating new skills, before deploying agents, or when debugging inconsistent agent behavior. Triggers on "evaluate", "test quality", "is this skill working", or QA of AI workflows.
The English summary is being prepared. Showing the original description.
Use when analyzing server logs to understand how AI crawlers (GPTBot, ClaudeBot, PerplexityBot) interact with your site. Use when optimizing content placement for LLM retrieval, diagnosing why AI search isn't citing your content, or auditing crawl patterns to find optimization gaps.
The English summary is being prepared. Showing the original description.
Créez des storyboards visuellement cohérents en utilisant la technique des 2x2 Grid Shots de PJ Ace, garantissant éclairage, personnages et décors uniformes entre les plans. Use when: **Après avoir finalisé un script vidéo** - Transformer le concept en visuels; **Besoin de cohérence visuelle** - Personnages et éclairage constants entre les plans; **Préparer des assets pour animation** - Frames prêtes pour Veo, Runway, Kling; **Présenter un storyboard client** - Visualisation avant production;...
The English summary is being prepared. Showing the original description.