본문으로 건너뛰기
cccskills
무료GitHub에 공개

ab-test-setup-and-analysis

Plan and analyze A/B tests for Google and Meta advertising. Estimate required traffic and duration, track results, and assess whether observed differences are statistically significant, meaning unlikely to be explained by chance alone.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

원문원문 설명 보기

Designs statistically valid split tests for ads, audiences, landing pages, or bid strategies. Calculates required sample sizes before you start, monitors results during the test, and calls winners when statistical significance is actually reached — not when you feel like one is winning. Platform: Google and Meta.

실행 점수 미검증

설치 방법 보기

이럴 때 유용합니다

  • Planning a test for a new ad headline
  • Estimating traffic and time needed for a landing page test
  • Checking whether interim results justify choosing a winner
  • Deciding what to roll out or test next

포함된 파일(1)

  • SKILL.md2.9 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

28/ A/B Test Setup and Analysis — Google + Meta

What it does

Designs statistically valid split tests for ads, audiences, landing pages, or bid strategies. Calculates required sample sizes before you start, monitors results during the test, and calls winners when statistical significance is actually reached — not when you feel like one is winning.

How it works

Claude takes your test hypothesis, current baseline metrics, and the minimum detectable effect you care about, then calculates how much traffic and time the test needs to produce a reliable result. During the test, it tracks results and tells you whether differences are statistically significant or just noise. It prevents premature test calls that waste the effort.

Practical example

You want to test a new headline variant against your current best performer. Your current ad gets a 2.8% CTR with 1,200 daily impressions. Claude calculates that to detect a 15% improvement (3.22% CTR) with 95% confidence, you need approximately 12,400 impressions per variant — about 10 days at current traffic levels. After 7 days, the new variant shows 3.1% CTR vs 2.7% for control. Claude tells you it's trending positive but not yet significant (p=0.14, need p<0.05) — keep running for 4 more days before making a call.

What you get back

  • Test design with clear hypothesis, control, variant, and success metric
  • Required sample size and estimated test duration before you start
  • Daily monitoring with current results, confidence level, and days remaining
  • Winner call only when significance is reached, with effect size and confidence interval
  • Post-test recommendations: roll out winner, iterate further, or test something else

When to use it

  • Before launching any creative, audience, or landing page test
  • When someone on the team wants to call a winner after 2 days of data
  • For structured testing programs where you need a pipeline of tests running sequentially
  • When clients ask "is this result real or just noise" and you need a definitive answer

Data access (Ryze MCP)

This skill works best with live account data. Connect the free Ryze MCP once and Claude reads your Google Ads, Meta Ads, GA4 and Search Console directly:

  • claude.ai / Claude Desktop: Settings → Connectors → Add custom connector → https://connector.get-ryze.ai/mcp
  • Claude Code: claude mcp add ryze --transport http https://connector.get-ryze.ai/mcp
  • Cursor: Settings → MCP → add the same URL

Setup guide: https://www.get-ryze.ai/how-to-connect-claude-to-google-meta-ads-mcp

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Analyze A/B test results to assess whether observed differences are likely due to chance. Check test duration and sample size, compare audience segments, distinguish statistical findings from practical value, and generate hypotheses for follow-up tests.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Evaluating results from a headline A/B test
  • Estimating sample sizes before starting a test
  • Comparing results by device or traffic source
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

Review Google and Meta campaign structures against goals, budgets, and conversion data. Identify excessive segmentation, overlapping targeting, and fragmented budgets, then get consolidation recommendations, an explained account structure, and a migration plan.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Reviewing an account inherited from another team or agency
  • Running a quarterly advertising account health check
  • Investigating structural causes of stalled performance
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

Analyze successful Google and Meta ads to find useful patterns in headlines, body text, and calls to action. Generate new copy variants that preserve those patterns, with platform formatting, test priorities, and suggested pairings for comparison.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Refreshing ad copy when existing creatives lose effectiveness
  • Starting an ad copy brainstorming session
  • Testing messaging angles for a new audience
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

Review Google Ads extensions, the extra links and details shown with ads, across campaigns. Find missing coverage, weak performance, and outdated information, then get replacement copy based on successful ads and landing pages, plus rollout priorities.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Checking extension coverage before launching campaigns
  • Quarterly reviews of outdated promotional details
  • Updating extensions after website changes or product launches
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

Analyze spending, revenue, and conversions across advertising channels to recommend budget shifts. Compare overall efficiency with the revenue gained from additional spending, identify saturation, and produce a gradual reallocation plan with monitoring priorities.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Reviewing budget allocation across advertising channels
  • Finding channels with declining returns on extra spending
  • Balancing proven campaigns with new experiments
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

Build a question set to check whether AI assistants mention your brand. Analyze the answers you collect, compare visibility against competitors, trace cited sources, and receive a prioritized list of opportunities to address gaps.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Establishing a baseline for brand visibility in AI answers
  • Comparing brand mentions and citations with competitors
  • Tracking visibility changes each quarter
irinabuht12-oss/marketing-skills5.6천2026년 10월 23일 업데이트

irinabuht12-oss의 스킬 모두 보기

이 스킬의 문제 신고