1.3K skills · sorted by relevance
Turn a research experiment plan into training and evaluation code, run a small setup check, and launch GPU experiments. Collect initial metrics and update progress records so the results are ready for further review.
Turn a vague research direction into a focused proposal and a detailed experiment roadmap. This workflow connects method refinement with validation planning, producing review records, an experiment tracker, and a summary of what to run next.
A platform decision framework for experimentation. When to use Statsig vs PostHog vs GrowthBook vs Optimizely vs Amplitude vs Eppo vs Kameleoon. How to migrate between them. How to coordinate when multi-platform is genuinely warranted. The decisions that compound for years and the ones you can defer. Triggers on which experimentation platform, choose Statsig vs PostHog, evaluate experimentation tools, switch experimentation platform, migrate from Optimizely, consolidate experimentation tools, multi-platform experimentation, experimentation platform decision, ab test platform selection, feature flag platform vs experiment platform, warehouse-native experiments, vendor lock-in experimentation. Also triggers when a team is asking about cost, governance, or migration cost across experimentation tools, or when an evaluation is starting.
The English summary is being prepared. Showing the original description.
Plan A/B tests to compare page changes using clear hypotheses and meaningful measures. The skill helps define sample size, duration, traffic allocation, and result interpretation, and organize a continuing program of prioritized experiments.
Resolves a PostHog experiment reference from natural language to a concrete experiment ID by browsing `experiment-list` (not feature-flag tools), with disambiguation when multiple experiments match. Use when the user names or quotes an experiment ("split test demo", "the File engagement boost experiment", "onboarding retention test", "landing page hero experiment", "pricing experiment"), describes it loosely ("the signup experiment", "my pricing test", "the one with the new checkout"), uses a relative reference ("latest", "most recent", "the one I created yesterday"), filters by status (running, draft, stopped, archived), or otherwise refers to an experiment by anything other than its concrete ID.
The English summary is being prepared. Showing the original description.
Running experiments out of the data warehouse instead of via dedicated experiment platforms. SQL-based assignment, exposure logging discipline, metric definitions in dbt models, statistical analysis in SQL or Python, variance reduction with CUPED, sequential testing, and the operational tradeoffs vs platforms like Statsig and Optimizely. Triggers on warehouse-native experimentation, run experiments in BigQuery, run experiments in Snowflake, dbt experiments, SQL t-test, CUPED variance reduction, exposure log, sample ratio mismatch, sequential testing, mSPRT, doubly robust estimation, build vs buy experimentation. Also triggers when the team is choosing between platform and warehouse, building warehouse-native experiment infrastructure, auditing one, or running an experiment with a custom metric the platform cannot handle.
The English summary is being prepared. Showing the original description.
Plan quantitative research by comparing experiments and surveys against research goals and resource limits. Get design recommendations, sample size calculations, participant recruitment strategies, and experimental materials, with an assessment of factors that could distort results.
When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.
The English summary is being prepared. Showing the original description.
When the user wants to plan, design, or implement an A/B test or experiment, or build a growth experimentation program. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "should I test this," "which version is better," "test two versions," "statistical significance," "how long should I run this test," "growth experiments," "experiment velocity," "experiment backlog," "ICE score," "experimentation program," or "experiment playbook." Use this whenever someone is comparing two approaches and wants to measure which performs better, or when they want to build a systematic experimentation practice. For tracking implementation, see analytics. For page-level conversion optimization, see cro.
The English summary is being prepared. Showing the original description.
Diagnoses bias, anomalies, and strange-looking results on a specific PostHog experiment. Covers empty / 0-exposure experiments, sample ratio mismatch, identity fragmentation, multi-variant exposure, uneven-split exclusion bias, significance traps (peeking, A/A, Bayesian vs Frequentist), PostHog-vs-SQL discrepancies, and surprises after mid-run edits. Symptom-driven dispatch to the right diagnostic. TRIGGER when: user asks 'is my experiment biased?' or 'why 0 exposures?', references the bias banner, says a variant looks strange / wrong / off, sees significance flipping, notices PostHog numbers disagreeing with their SQL, sees an A/A test showing significance, or reports surprises after mid-run edits. DO NOT TRIGGER when: creating a new experiment (use creating-experiments), only configuring rollout (use configuring-experiment-rollout) or metrics (use configuring-experiment-analytics), or only asking lifecycle questions (use managing-experiment-lifecycle).
The English summary is being prepared. Showing the original description.
Orchestrator for the experimentation skill suite — turn assumptions and product questions into rigorous, well-instrumented experiments and decision-grade readouts. Routes through backlog → spec → runbook → readout based on user need and existing artefacts. Platform-agnostic with PostHog as the primary binding. Load when the user asks to design an experiment, A/B test something, set up an experiment, run a holdout, test a hypothesis, decide what to test next, read out experiment results, analyse a test, or says "should we A/B test this", "experiment on the landing page", "is this lift real", "ship or kill this test", "what should we test next", "build an experiment backlog", "test the pricing page", "validate this with an experiment".
The English summary is being prepared. Showing the original description.
Create and analyze AI model evaluation experiments in Arize using the ax CLI, a command-line tool. Collect real model outputs, export results, compare evaluation scores, and identify examples where a new model performs worse.
Plan controlled failure experiments to uncover system weaknesses. Define normal operating metrics, estimate affected users and reliability risk, set abort and recovery conditions, and turn experiment results into a structured review with follow-up actions.
Turn validated experiment analysis into an internal research report that supports decisions. Organize findings, statistical evidence, figure interpretations, limitations, and next actions, then save the report as Markdown or write it into a connected Obsidian knowledge base.
Turn completed experiment analyses into an internal research report that explains findings, figures, limitations, and next actions. Reports connect evidence to decisions and can be saved locally or written into a linked Obsidian project.
Plans, builds, runs, and analyzes fault injection experiments with AWS Fault Injection Service (AWS FIS) to validate application resilience through chaos engineering. Covers experiment templates (actions, targets, stop conditions), the FIS actions catalog and action selection, the scenario library (AZ power interruption, cross-Region connectivity, EC2/EKS/EBS stress), experiment lifecycle and monitoring, logging and reports, multi-account experiments, the experiment IAM role, and blast-radius safety. Applies when a user mentions AWS FIS or fault injection, asks what experiment to run for a failure mode (AZ, Region, API errors/throttling, instance/pod/DB/cache failure, latency, packet loss), wants to author an experiment template or CLI/CloudFormation, or needs to safely run chaos experiments in pre-production or production. For the broader resilience program across Resilience Hub and ARC, see aws-resilience-lifecycle; for ARC routing controls and zonal shift, see recovery-controller-setup.
The English summary is being prepared. Showing the original description.
Operator-side analysis of historical ML experiment PRs in Senpai research tracks. Use this skill whenever the user asks to: analyze experiments, categorize PRs, bucket experiments, summarize what's been tried, understand experiment history, review merged vs closed results, or asks "what experiments have we run / worked / failed". Also triggers for: "pull the latest experiments", "what's been tried so far", "category breakdown of PRs", "which experiments succeeded", "noam track analysis". When a branch name is mentioned (e.g. "noam branch", "on the noam branch"), pass it as the base branch to scope the fetch to just those PRs.
The English summary is being prepared. Showing the original description.
Guides agents through the 3-step experiment creation flow: defining the hypothesis, configuring rollout, and setting up analytics. Delegates rollout decisions to configuring-experiment-rollout and metric setup to configuring-experiment-analytics. TRIGGER when: user asks to create a new experiment or A/B test, OR when you are about to call experiment-create. DO NOT TRIGGER when: user is updating an existing experiment, managing lifecycle, or only browsing experiments.
The English summary is being prepared. Showing the original description.
Organize experiment photos, audio, and notes into Markdown logs with structured metadata. Assign experiment and sample batch IDs, archive original attachments, and optionally connect records to Feishu group messages or an Obsidian notebook.
Improve code through repeated experiments against a measurable target, such as execution time or memory use. After defining scope and constraints, the agent tests changes, keeps improvements, reverts unsuccessful attempts, and logs results.
Launch machine learning experiments on local GPUs, remote servers, Vast.ai, or Modal. It checks the configured environment, deploys code, saves logs, and verifies startup, with optional W&B metric logging and automatic Vast.ai result collection and instance cleanup.
Check experimental results before making research claims. A fresh reviewer examines reference data, score calculations, result files, and evaluation scope, then produces an evidence-based report showing which claims are supported and which need qualification.
Run batches of machine learning experiments on remote GPUs over SSH, varying settings and random seeds. The skill schedules jobs, tracks progress, retries GPU memory failures, and cleans up leftover sessions before starting subsequent batches.
Turn a research proposal into an experiment plan tied to specific paper claims. Define comparisons, evaluation measures, tests that remove individual components, run order, and compute estimates, with a written plan and tracker for execution.