Skip to content
cccskills
FreePublished on GitHub

ab-test-setup

Design, analyze, and document A/B tests for conversion, onboarding, pricing, lifecycle, and product experiments. Use when the user asks for `/ab-test-setup`, experiment design, sample size, statistical significance, A/B test analysis, ICE-scored test backlogs, or avoiding common testing mistakes.

The English summary is being prepared. Showing the original description.

See how to install

Files included (1)

  • SKILL.md2.3 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

A/B Test Setup

Overview

Use this skill to guide the full experiment lifecycle: hypothesis, design, sample size, implementation, analysis, and playbook documentation. Keep tests focused, measurable, and resistant to common errors like peeking early or testing too many changes at once.

Workflow

  1. Define the business goal, user segment, current baseline, target metric, and guardrail metrics.
  2. Write a specific hypothesis:
    • If we change X for audience Y, metric Z will improve because...
  3. Design the test:
    • Control and variant.
    • Primary metric.
    • Secondary and guardrail metrics.
    • Traffic split, eligibility, exclusions, and duration.
  4. Estimate sample size or minimum detectable effect when baseline traffic and conversion rates are available.
  5. Create an implementation checklist:
    • Tracking, randomization, QA, exposure logging, analytics events, and rollback.
  6. Define decision rules before launch:
    • Ship, revert, iterate, or continue testing.
  7. Analyze results after the test reaches the agreed sample size.
  8. Document what changed, what was learned, and follow-up experiments.

Test Backlog Pattern

When building a backlog, score each idea with ICE:

  • Impact: expected business or user benefit.
  • Confidence: evidence quality.
  • Effort: complexity and implementation cost.

Prioritize tests that combine high impact, credible evidence, and low operational risk.

Example Prompts

  • I want to A/B test our signup CTA button. Current conversion rate is 3.2%, 8,000 visitors/month. Help me design the test, calculate the required sample size, and define what success looks like.
  • Our A/B test just hit sample size. Here are the results [paste metrics]. Is this statistically significant? Should we ship the variant, revert, or keep testing?
  • Build a prioritized A/B test backlog for our onboarding flow. Use ICE scoring. Sources to mine: our drop-off analytics, last month's support tickets, and these 3 heatmap observations.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Creating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems. Create original algorithmic art rather than copying existing artists' work to avoid copyright violations.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

Create beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

Transform git commit history into polished user-facing changelogs and release notes. Use when the user wants a CHANGELOG.md from a version tag or date range, weekly product updates, app store release notes, or Keep a Changelog formatted output with internal noise filtered out.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

Full legal plugin suite for commercial contract review, privacy/DPA analysis, and AI governance vendor reviews. Use when the user invokes /commercial-legal:review, /privacy-legal:dpa-review, /ai-governance-legal:vendor-ai-review, renewal tracking, DSAR responses, or EU AI Act triage.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

Review contracts with clause-by-clause risk scoring, market benchmarks, negotiability ratings, and redline suggestions. Use when the user pastes an NDA, SaaS/MSA, M&A LOI, or payment agreement and wants a Contract Safety Score, CUAD-based risk breakdown, or /review as [position] analysis.

The English summary is being prepared. Showing the original description.

eigent-ai/agent-skills24Updated Oct 14, 2026

See all skills by eigent-ai

Report a problem with this skill