Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
The English summary is being prepared. Showing the original description.
See how to installReview the instructions the agent will receive before you install.
Use this skill for analysis over graded JSONL outputs, typically after:
agent-eval-harness eval '{"mode":"grade",...}' > graded.jsonl
For normal baseline-vs-challenger comparisons, use:
agent-eval-harness eval '{"mode":"compare",...}'
Use custom scripts only when suite-specific analysis is required.
When writing a custom script:
trial_result.pass and score non-null).taskId, trialIndex, runId).exactPassAtK naming for combinatorics-based pass@k metrics (do not label exact values as estimatedPassAtK).Reference helpers remain in references/ for bootstrap/statistics utilities.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
The English summary is being prepared. Showing the original description.
Use test-driven development for behavior-changing feature or fix work, and whenever the user mentions TDD, test-first, red-green-refactor, tracer bullets, integration tests, or public-interface behavior tests. Skip for docs-only, path-only rename, formatting-only, or purely mechanical chores unless explicitly requested.
The English summary is being prepared. Showing the original description.
Build command-only adapters for `eval` run mode using strict stdin/stdout JSON contracts.
The English summary is being prepared. Showing the original description.
End-to-end eval-suite orchestration with the `eval` command: run -> grade -> compare -> calibrate, using strict JSON contracts and schema discovery.
The English summary is being prepared. Showing the original description.