compare-trials
FreeAnalyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
The English summary is being prepared. Showing the original description.
Use test-driven development for behavior-changing feature or fix work, and whenever the user mentions TDD, test-first, red-green-refactor, tracer bullets, integration tests, or public-interface behavior tests. Skip for docs-only, path-only rename, formatting-only, or purely mechanical chores unless explicitly requested.
The English summary is being prepared. Showing the original description.
See how to installReview the instructions the agent will receive before you install.
Work in vertical red-green-refactor slices:
Do not write a batch of imagined tests before implementation. Each new test should respond to what the last cycle taught you about the actual behavior and interface.
bun test for targeted tests and bun --bun tsc --noEmit for the type gate.test, not it, and organize with describe.AGENTS.md rules for the touched area.Explore the codebase first. If the public interface, expected behavior, or risk boundary is discoverable from code, tests, or docs, use that evidence instead of asking the user.
Ask one concise question only when the decision cannot be discovered and a reasonable assumption would be risky. Include your recommended answer.
Before writing production code, identify:
For each behavior:
If the test passes before production code changes, it is not a valid RED signal. Tighten the test, choose a different behavior, or explain why existing coverage already proves the behavior.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
The English summary is being prepared. Showing the original description.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
The English summary is being prepared. Showing the original description.
Build command-only adapters for `eval` run mode using strict stdin/stdout JSON contracts.
The English summary is being prepared. Showing the original description.
End-to-end eval-suite orchestration with the `eval` command: run -> grade -> compare -> calibrate, using strict JSON contracts and schema discovery.
The English summary is being prepared. Showing the original description.