Analyze A/B test results to decide whether to launch a change, extend testing, stop, or investigate further. Check sample size, statistical significance, uncertainty ranges, and effects on other important metrics, then summarize the recommendation and next steps.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Analyze A/B test results with statistical significance, sample size validation, confidence intervals, and ship/extend/stop recommendations. Use when evaluating experiment results, checking if a test reached significance, interpreting split test data, or deciding whether to ship a variant.
설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.
Evaluate A/B test results with statistical rigor and translate findings into clear product decisions.
You are analyzing A/B test results for $ARGUMENTS.
If the user provides data files (CSV, Excel, or analytics exports), read and analyze them directly. Generate Python scripts for statistical calculations when needed.
Understand the experiment:
Validate the test setup:
Calculate statistical significance:
If the user provides raw data, generate and run a Python script to calculate these.
Check guardrail metrics:
Interpret results:
| Outcome | Recommendation |
|---|---|
| Significant positive lift, no guardrail issues | Ship it — roll out to 100% |
| Significant positive lift, guardrail concerns | Investigate — understand trade-offs before shipping |
| Not significant, positive trend | Extend the test — need more data or larger effect |
| Not significant, flat | Stop the test — no meaningful difference detected |
| Significant negative lift | Don't ship — revert to control, analyze why |
Provide the analysis summary:
## A/B Test Results: [Test Name]
**Hypothesis**: [What we expected]
**Duration**: [X days] | **Sample**: [N control / M variant]
| Metric | Control | Variant | Lift | p-value | Significant? |
|---|---|---|---|---|---|
| [Primary] | X% | Y% | +Z% | 0.0X | Yes/No |
| [Guardrail] | ... | ... | ... | ... | ... |
**Recommendation**: [Ship / Extend / Stop / Investigate]
**Reasoning**: [Why]
**Next steps**: [What to do]
Think step by step. Save as markdown. Generate Python scripts for calculations if raw data is provided.
아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.
로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.
아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.
Organize customer feature requests into themes and rank the top three against product goals, customer value, effort, and risk. For each recommendation, explain the underlying need, possible alternatives, uncertain assumptions, and simple ways to test them.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Compare growth options across existing and new products and markets using the Ansoff Matrix, a four-part planning framework. Assess opportunities, resources, competition, and risks, then develop market launch plans and a phased roadmap for priority options.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Choose the first customer segment for a product launch by comparing urgent needs, willingness to pay, achievable market share, and referral potential. Get scored recommendations, a priority segment, a 90-day acquisition plan, and expansion options.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Plan low-effort experiments to check assumptions about an existing product before building a feature. Compare prototypes, A/B tests, and other methods, with a clear assumption, procedure, metric, and success threshold for each experiment.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Turn a new product idea into a testable hypothesis and two or three simple experiments. Define target customers, expected actions, measurement methods, and success thresholds to assess demand through behavior such as sign-ups, pre-orders, or paid commitments.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Generate improvement ideas for an existing product from product management, design, and engineering perspectives. Produce five ideas per perspective, then select the top five based on goals, potential impact, feasibility, and differentiation, with reasons and assumptions to test.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.