Use when about to claim a result, effect, significance, or that an analysis reproduces, before reporting or writing it up - requires running the analysis fresh and reading the actual output first; evidence before claims always
The English summary is being prepared. Showing the original description.
See how to installReview the instructions the agent will receive before you install.
Claiming a finding without fresh verification is dishonesty, not efficiency.
Core principle: Evidence before claims, always.
Violating the letter of this rule is violating the spirit of this rule.
NO CLAIMS WITHOUT FRESH REPRODUCED EVIDENCE
If you haven't run the analysis in this state and read its actual output, you cannot claim its result. "It was significant earlier" is not evidence now.
BEFORE claiming any result or expressing satisfaction:
1. IDENTIFY: What command/analysis proves this claim?
2. RUN: Execute it fresh and complete (from the immutable raw data, fixed seed)
3. READ: The actual output — the estimate, the interval, the p-value, the diagnostics
4. CHECK: Do the method's assumptions hold? Does it reproduce?
For a CONFIRMATORY claim: does the pre-registration audit pass?
(prereg.sh, ships with science-superpowers:preregistering-analysis:
<skills root>/preregistering-analysis/prereg.sh audit)
5. VERIFY: Does the output actually support the claim?
- If NO: state the real result with evidence
- If YES: state the claim WITH the evidence (number + interval)
6. ONLY THEN: make the claim
Skip any step = asserting, not verifying
| Claim | Requires | Not Sufficient |
|---|---|---|
| "The effect is significant" | Fresh run; read estimate, CI, and p | "It was significant before" |
| "There's no effect" | Effect size + interval showing precision | A non-significant p (could be underpowered) |
| "The result reproduces" | Re-run from raw data + fixed seed → same number | "It ran fine earlier" |
| "Assumptions are met" | The diagnostic output, read | "It's probably fine" |
| "The model is good" | Out-of-sample metric | In-sample fit / training accuracy |
| "Data cleaned correctly" | Validation counts (rows in/out, ranges) | "The script ran without error" |
| "Confirmatory finding" | prereg.sh audit passes AND re-run as registered | The prereg file was committed before the results (it may have been edited after) |
| "The subagent finished" | Inspect the committed artifacts/diff | The subagent said "done" |
prereg.sh audit — commit order alone is not a freeze| Excuse | Reality |
|---|---|
| "It was significant last run" | Re-run it now. Code/data may have changed. |
| "I'm confident in the result" | Confidence is not evidence. |
| "The script ran without errors" | Running ≠ correct. Read the output. |
| "p < .05, so it's real" | p is not the probability the effect is real. Report effect + interval. |
| "p > .05, so no effect" | Could be underpowered. Absence of evidence ≠ evidence of absence. |
| "The subagent reported success" | Verify the artifacts independently. |
| "The prereg was committed before the results, so it's frozen" | Commit order misses post-freeze edits to the registration. Run prereg.sh audit; read the frozen-vs-current diff. |
| "It reproduces, I'm sure" | Re-run from raw + seed and show the same number. |
| "Different words, so the rule doesn't apply" | Spirit over letter. |
Significance / effect:
✅ [Re-run the model] [Read: beta=0.23, 95% CI [0.08, 0.38], p=.002] "Exposure raises the outcome; CI excludes zero."
❌ "The effect looked significant."
Reproducibility:
✅ Fresh env → re-run from data/raw with seed → same estimate to reported precision → "Reproduces."
❌ "It ran earlier, so it reproduces."
Null result:
✅ [Read: beta=0.01, 95% CI [-0.12, 0.14]] "No detectable effect; the interval rules out effects larger than ~0.14."
❌ "p=0.3, so there's no effect."
Model performance:
✅ [Held-out test set, used once] [AUC=0.78] "Out-of-sample AUC 0.78."
❌ "Training accuracy is 0.97, the model is great."
Confirmatory label:
✅ [Run prereg.sh audit] [Read: RESULT: PASS] → apply the frozen decision rule to the fresh output
❌ "The pre-registration predates the results commit, so the finding is confirmatory."
Subagent delegation:
✅ Subagent reports done → inspect committed code + output artifact → confirm → report actual state
❌ Trust the report
ALWAYS before:
Applies to: exact phrases, paraphrases, synonyms, and any implication of a verified finding.
Run it fresh. Read the output. Check it reproduces. THEN state the result — with the number and the interval.
This is non-negotiable.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Use when you have an approved research question and need a concrete analysis plan, before touching outcome data or fitting any model
The English summary is being prepared. Showing the original description.
Use when facing 2+ independent investigations that can proceed without shared state - parallel literature survey, multi-dataset replication, or pre-specified robustness checks
The English summary is being prepared. Showing the original description.
Use when your human partner has explicitly opted into exploratory or feasibility mode - a compute-heavy simulation, an unproven pipeline, an unbenchmarked solver, an untested cluster job - or when whether the work can run at all is still unknown, when a plan's largest configuration has never been executed, when a memory or wall-clock ceiling is estimated rather than measured, or when someone proposes pre-registering over runs that have never happened
The English summary is being prepared. Showing the original description.
Use when you have a pre-registered analysis plan to execute inline in this session with review checkpoints, on a platform without subagents
The English summary is being prepared. Showing the original description.
You MUST use this before any data analysis or investigation - before exploring a dataset, loading or profiling data, running a model, computing a statistic, or testing an idea, and before any outcome data is touched
The English summary is being prepared. Showing the original description.
Use when a result is surprising, impossible, contradicts a sanity check, a pipeline fails, a model won't converge, or a replication fails - before adjusting anything
The English summary is being prepared. Showing the original description.