eval

Run one evaluation of a Library skill and show its board. Eval never mints a version and never moves skill bytes. When the evaluated bytes are already a published version it publishes that run's receipt to the team (--no-commit keeps it on this machine); otherwise it prints a share hint.

Arguments: everything after the command (in Claude Code this arrives as "$ARGUMENTS"): one skill name (or nothing when the working directory is inside a Library skill folder) or several, then any of --k <n>, --triggers-only, --execution-only, --case <stem>, --model <m>, --judge-model <m>, --no-gen, --no-commit (keep the receipt on this machine instead of publishing it to the team), --parallel <n>, --pending (every shared skill with no receipt for its current version), --window overnight|later (queue instead of running; nothing is paid), passed through unchanged. Leave --batch <n> out: it asks "Continue?" before each further batch, which needs a terminal. The queue flags (--queue-list, --dequeue <skill>, --drain …) belong to the terum-skills skill; --dequeue and --drain need a confirmation first.

Command

npx -y terum-skills@latest eval <skill> [flags] --format md

Run it in the background when your shell tool can, or with a long timeout: a full matrix takes minutes. Show the board when it finishes.

Before

Confirm with the user first: trials and generation run their logged-in Claude Code and bill their account. State what the run buys — cases × k × arms (k defaults to 1; two arms, three when a published incumbent exists) plus generation calls for any missing cases or triggers, times the number of skills for a batch (several skills run as one batch after one preflight, --parallel <n> at a time, default four; confirm once for the whole batch) — and, when the skill-info board shows a previous run, quote its cost and duration as the estimate. --window overnight|later queues instead of running and costs nothing now. Suggest --triggers-only or --case <stem> for a first look. What eval does, so the user knows:

1. Finds the local folder by name; it need not belong to a team. When a team is configured, the clone is fetched for policy and incumbent history, never for the candidate bytes.
2. Runs hygiene on the local bytes, allowing absent managed fields; errors stop before any agent call.
3. Preflight checks claude --version and a one-turn smoke task before paid trials.
4. Uses existing cases and triggers, generating only the missing assets for the requested tiers; it prints the target path and the content-change consequence before writing them into the folder. Generating assets changes content identity and can mint a version on the next publish; --no-gen uses only existing assets, and regenerating cases means deleting evals/cases/ and re-running. The generated bytes are re-checked against HYG2/HYG3 (the same predicates and author exemption the next command uses) before the write-back; a failure there refuses the run and writes nothing.
5. Runs trigger evals and execution cases, each k times, comparing the candidate with the baseline and, when available, a digest-different published incumbent chosen by receipt recency.
6. Stores receipts and run artifacts under ~/.terum/skills/evals/local/<digest>/<run-id>/ and prints the report; a later publish attaches receipts whose digest matches the published bytes; when these exact bytes are already a published version the receipt goes to the team now (--no-commit keeps it local).

After

Show the board verbatim: the verdict band, the comparison table and any ⚠ marks stay as printed; never average holes into a clean number. Already evaluated these exact bytes is a result — the earlier receipt stands. A batch shows one board with Evaluated X of N and the queued remainder; offer the eval-report skill for each skill afterwards. Model flags pass through unchanged (--model, --judge-model; default sonnet). Editing the skill changes its digest, so the Library's score belongs to the bytes that were evaluated; installed receipts keep their original runner and are not proof this user ran anything. The one exit-1 that does ask for a re-run is the generated-assets hygiene refusal, which says so in its own text.

Rules

• No TTY: never pipe y, never drive the CLI with expect, never add --frames to dodge a question. A question the CLI asks means the verb belongs in a terminal: say so and hand over the command.
• Never use the skill-file !`command` injection; run every command with your shell tool and read its output.
• Do not cd; run from the current working directory and pass absolute paths.
• Exit 1 is a result, not a retry: show the failure block, do not re-run, and do not claim earlier steps were rolled back.

Sandbox

When CODEX_SANDBOX_NETWORK_DISABLED=1 is set and the command starts with npx, add --prefer-offline after npx so a cached package resolves without the registry; if npx still reports a network error, ask the user to run the command in a terminal. In that sandbox the verbs that need the network — sync, install, publish, invite, eval, and update's release probe — are handed to a terminal with the reason.