End-to-end paper reproduction from arxiv URL through smoke runs to replication experiments. Handles missing or partial official code, missing training scripts, missing hyperparameters, and private datasets via similar-public-dataset substitution. Use when the user asks to reproduce, implement, replicate, or re-run a paper from scratch, or pastes an arxiv URL with reproduction intent.
インストールする前に、エージェントに与えられる指示の中身を確認できます。
Reproducing an ML paper often means filling gaps the authors didn't ship, training scripts, hyperparameter tables, augmentation specifics, exact dataset splits. This skill walks seven stages from "I have an arxiv link" to "I have a replication run with measurable delta vs the paper's number."
Each stage has a separate reference file under references/ so this overview stays scannable.
The user just said any of:
| Stage | What | Reference |
|---|---|---|
| 1 | Paper acquisition (CLI + PDF → structured extract) | references/01-paper-fetch.md |
| 2 | Existing code discovery + inventory | references/02-code-clone.md |
| 3 | Gap analysis (extract every missing hyperparam from the prose) | references/03-gap-analysis.md |
| 4 | Implementation (uv venv, fill gaps, commit per gap) | references/04-implement.md |
| 5 | Dataset acquisition (HF datasets first; substitute if private) | references/05-dataset.md |
| 6 | Smoke runs (forward pass → 1 step → 20 iters) | references/06-smoke.md |
| 7 | Replication runs + comparison at paper's reported epochs | references/07-replicate.md |
Walk them in order. Each stage has its own success criteria; do not advance to the next until the current one passes.
For each paper reproduction, set up a dedicated workspace:
repro/<paper-arxiv-id>/
├── paper.md # structured extract from stage 1
├── inventory.md # what exists / missing from stage 2
├── gaps_filled.md # hyperparam table with provenance from stage 3
├── code/ # implementation from stage 4 (or cloned + extended)
├── data/ # dataset symlinks or actual data from stage 5
├── dataset_substitution.md # if a public dataset stood in for a private one
├── smoke_logs/ # outputs from stage 6
└── results.md # replication outcomes from stage 7
This keeps reproductions self-contained and easy to revisit later.
paper-verification skill for a round-trip check ("did I really capture every hyperparam the paper mentions")./phd-skills:debug skill, not to ad-hoc fixes./phd-skills:launch checklist before any multi-hour run./phd-skills:compare skill at the paper's reported epochs (never current-vs-final).For each reproduction, the final artifact is results.md with absolute deltas (not just %) and one of three labels per metric:
[matched within 0.X pp]: within the paper's reported variance[gap, hypothesis: ...]: measurable underperformance, with a stated hypothesis for the cause[fundamental disagreement, see X]: the result and the paper's claim are inconsistent in a way that needs investigation, not just more computeIf the workspace is on a public repo, link the workspace README from the project's main reproduction-tracking doc.
まだレビューはありません。使ってみた感想をお寄せください。
概要と使いどころ
Same-epoch comparison of training runs across wandb, neptune, tensorboard, or mlflow. Aligns runs at the student's current step (never current-vs-final-of-baseline) and separates proxy metrics from downstream targets. Use when the user asks to compare runs, check if a run is improving, track lag against a baseline, rank experiments, or evaluate run-vs-run performance.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants to analyze dataset bias, create stratified samples, evaluate fairness, or plan dataset collection. Triggers on phrases like "dataset bias", "stratified sample", "class imbalance", "data distribution", "fairness analysis", or "ethical review".
日本語の概要は準備中です。原文の説明を表示しています。
Evidence-before-action diagnosis of failing ML experiments. Probes the system before guessing causes, process list, dmesg, GPU stats, log scrollback, checkpoint state, then states a hypothesis as a hypothesis and runs a smoke before claiming a root cause. Use when the user asks why a run is failing, diverging, OOMing, hanging, slow, producing weird metrics, has crashed, or asks to debug, diagnose, troubleshoot, or investigate a training issue.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants to design experiments, plan ablation studies, structure baselines, or create incremental evaluation strategies. Triggers on phrases like "design ablation", "plan experiment", "what experiments should I run", "baseline comparison", or "experiment matrix".
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants to set up or troubleshoot a LaTeX environment, choose between biber and bibtex, install packages for a specific venue template, or configure compilation. Triggers on phrases like "setup latex", "biber vs bibtex", "latex compilation error", "install latex packages", "venue template", or "texlive setup".
日本語の概要は準備中です。原文の説明を表示しています。
Pre-flight checklist for long-running ML training jobs covering config diff, run naming, path verification, monitoring setup, and restart-cleanup. Use when the user asks to launch, kick off, start, restart, or kill a training run, or mentions launching a multi-hour or multi-day GPU job (python train, accelerate launch, torchrun, deepspeed, sbatch, tmux training).
日本語の概要は準備中です。原文の説明を表示しています。