Use when the user wants to verify that the evolved agent's score is stable and reliable. Runs evaluation multiple times and reports mean ± std.
The English summary is being prepared. Showing the original description.
See how to installReview the instructions the agent will receive before you install.
Verify score stability by running evaluation multiple times and reporting statistical confidence.
TOOLS="${EVOLVER_TOOLS:-$([ -d ".evolver/tools" ] && echo ".evolver/tools" || echo "$HOME/.evolver/tools")}"
EVOLVER_PY="${EVOLVER_PY:-$([ -f "$HOME/.evolver/venv/bin/python" ] && echo "$HOME/.evolver/venv/bin/python" || echo "python3")}"
Read .evolver.json to get the best experiment and dataset.
Run evaluation 3 times on the current code (not a worktree — the best code is already merged):
for i in 1 2 3; do
$EVOLVER_PY $TOOLS/run_eval.py \
--config .evolver.json \
--worktree-path "." \
--experiment-prefix "certify-run-$i"
done
After all 3 runs complete, read results and compute statistics:
$EVOLVER_PY $TOOLS/read_results.py --experiments "certify-run-1-{suffix},certify-run-2-{suffix},certify-run-3-{suffix}" --config .evolver.json --format summary
Calculate mean and standard deviation from the 3 combined_scores.
CERTIFICATION REPORT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Runs: 3
Mean: {mean:.3f}
Std: {std:.3f}
Range: {min:.3f} — {max:.3f}
Verdict: {STABLE|UNSTABLE}
STABLE (std < 0.05): Score is reliable. The agent performs consistently.
MARGINAL (0.05 <= std < 0.10): Score varies moderately. Consider adding rubrics to reduce judge variance.
UNSTABLE (std >= 0.10): Score is unreliable. The LLM judge interprets criteria differently across runs. Add few-shot examples or tighter rubrics.
If STABLE: suggest /harness:deploy to finalize.
If UNSTABLE: suggest adding rubrics to dataset examples, or running /harness:evolve with heavy mode for more thorough evaluation.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Use when the user wants to smoke-test the evolve pipeline, test tools, or verify the plugin works end-to-end. Also use when the user says 'dry run', 'smoke test', or 'test pipeline'.
The English summary is being prepared. Showing the original description.
Use when the user wants to release a new version, publish to npm, create a GitHub release, bump version, or tag a release. Also use when the user says 'release', 'publish', 'bump version', 'tag', or 'npm publish'.
The English summary is being prepared. Showing the original description.
Use when the user wants to validate the plugin, check integrity, verify cross-references, or before a release. Also use when the user says 'validate', 'check plugin', or 'verify'.
The English summary is being prepared. Showing the original description.
Use when the user is done evolving and wants to finalize, clean up, tag the result, or push the optimized agent.
The English summary is being prepared. Showing the original description.
Use when the user wants to run the optimization loop, improve agent performance, evolve the agent, or iterate on quality. Requires .evolver.json to exist (run harness:setup first).
The English summary is being prepared. Showing the original description.
Use when the user wants to check dataset quality, diagnose eval issues, or before running evolve. Checks size, difficulty distribution, dead examples, coverage, and splits. Auto-corrects issues found.
The English summary is being prepared. Showing the original description.