Run matched baseline and treatment measurements for NeoHaskell framework flows, compiled test suites and consumer compilation, retaining workload counts, inputs and raw samples.
The English summary is being prepared. Showing the original description.
Run score Not tested
See how to installReview the instructions the agent will receive before you install.
Read the benchmark checklist before selecting a workload or reporting a result. Measure the affected command, decider, storage or downstream completion boundary. For derivation or generated API changes, include representative consumer compilation.
Write a protocol using the runner contract. Run python3 .agents/skills/framework-benchmark/scripts/compare.py <protocol.json>. The runner alternates baseline and treatment, keeps raw logs and records input hashes. It reports medians and ranges, without deciding whether the difference is meaningful.
Use a workload driver that independently checks the resulting state and emits completed work and error counts. Read that driver before measuring. Use a separate diagnostic run to identify the limiter. Compiled suite duration includes command startup and any build work. It is not request latency or framework throughput.
For compilation, separate clean, incremental and unchanged builds. Put each sample's cache preparation in the protocol's prepare command. Name the modules or splice work the sample must perform. An unchanged-build scenario may deliberately perform no compiler work. Keep dependencies, flags, edits and cache state equivalent across sides.
For scheduled suite-duration checks, run python3 .agents/skills/framework-benchmark/scripts/bench.py --strict. The suite registry owns selections, prerequisites, timeouts and optional calibrated budgets. A null budget records calibration data. It establishes no performance threshold.
Keep the protocol and outputs under repository-root .agents-state/framework-benchmark/. Report the actual completion boundary, errors, successful work, settings, samples, range and limiter. Follow the checklist's rules for an inconclusive result. The hypothetical throughput design target is not a required load-test size.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Sketch types, signatures, and module structure before code, then stay in the loop while implementation fills in. Use for /architect, 'architect this', 'design this', or non-trivial work where jumping to code would lock in the wrong shape.
The English summary is being prepared. Showing the original description.
Spawn N parallel candidates at the same task, pick a base, graft the strongest parts of the losers into it. Use for /arena, 'arena this', 'throw it in the arena', or when one attempt at a non-trivial artifact would lock in the wrong shape.
The English summary is being prepared. Showing the original description.
Inspect or supervise a PR or stack through review and CI to merge-ready, using current GitHub evidence and the requested mode.
The English summary is being prepared. Showing the original description.
Vet a perf measurement (limiter, tuning, limits, errors, repeatability, relevance, and whether the work happened) before you report or act on it. Use when you benchmark or report a measured speedup or regression, including complete framework flows and compilation of projects that consume the framework.
The English summary is being prepared. Showing the original description.
Restate the last message in plain human language, with no jargon.
The English summary is being prepared. Showing the original description.
Build or adapt a local harness to drive, inspect, and profile an interactive CLI or TUI without external services. Use for CLI UX checks, startup regressions, memory leaks, hangs, prompt flows, or terminal demos.
The English summary is being prepared. Showing the original description.