跳到正文
cccskills
免费在 GitHub 公开

evals-harness

Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns. Use when asked to "build an eval harness", "set up eval-as-code", or "version our eval datasets".

中文简介正在准备中。当前显示原文说明。

查看安装方法

包含的文件(1)

  • SKILL.md1.8 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Evals Harness

You are Evals — the LLM Evaluation Engineer on the AI Operations Team.

Steps

Step 0: Confirm Scope

Establish which models, prompts, or pipeline stages the harness needs to cover, and how often it needs to run (every PR, nightly, per release).

Step 1: Define Task Schema and Metrics

Specify the input/output schema for each task type and the metric(s) computed for it — exact match, rubric score, model-graded, or a domain-specific metric.

Step 2: Design Dataset Versioning and Eval-as-Code

Version the eval dataset alongside the code (not a spreadsheet someone edits by hand), and define the harness as a runnable, CI-invocable command with a clear pass/fail exit code.

Key Rules

  • Follow the output format defined in docs/output-kit.md
  • The harness must be runnable in CI, not just interactively — a harness that requires a human to eyeball results doesn't scale
  • Version the dataset — a harness whose ground truth silently changes underneath it produces meaningless trend lines

Output Format

A harness design — task schema, metric definitions, dataset versioning scheme, and how it plugs into CI.

Delivery

If output exceeds the 40-line CLI budget, invoke /atlas-report with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

apex

免费

Engineering lead — hand Apex any task and it routes internally. New features, planning, reviews, status, orientation, or system takeovers.

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

Session postmortem from local transcripts — why a run repeated work, ignored the plan, took too long, or cost more than expected. Use when asked "why did that take so long", "why was that so expensive", "what went wrong in that session", "why did the agent redo that", or when preparing a bug report about agent behavior.

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

apex-gate

免费

Inspect and tune the skill-manifest gate — which of the 421 tonone skills keep their description in this project's context, and what that costs in tokens. Use when asked "why can't Claude see this skill", "show the skill gate", "how many tokens do my skills cost", "trim the skill catalogue", or "undo the skill gate".

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

apex-plan

免费

Plan and scope a project — discovery, challenge assumptions, present XS-XXL depth options with token and cost estimates. Use when asked to "plan this", "scope this", "how should we build X", or when a new project/feature request comes in.

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

Scope the tonone agent roster for this project — install a curated subset of agents instead of the full 100-agent bundle. Use when "cut down the agent list", "profile for this project", "too many agents", "only need the engineering core", or after apex-stats shows a roster that's mostly unused.

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

Engineering lead reconnaissance — inventory the project before planning. Use when asked to "understand this project", "orient me on this codebase", "what's the state of the repo", "what's in progress", or before starting work on an unfamiliar codebase.

中文简介正在准备中。当前显示原文说明。

tonone-ai/tonone762026年10月15日 更新

查看 tonone-ai 的全部技能

报告此技能的问题