217 个 · 按相关度排序
Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies.
中文简介正在准备中。当前显示原文说明。
Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based tuple generation. Use when bootstrapping an eval dataset, when real user data is sparse, or when stress-testing specific failure hypotheses. Do NOT use when you already have 100+ representative real traces (use stratified sampling instead), or when the task is collecting production logs.
中文简介正在准备中。当前显示原文说明。
Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode requires interpretation (tone, faithfulness, relevance, completeness). Do NOT use when the failure mode can be checked with code (regex, schema validation, execution tests); use `write-code-eval`. To validate an existing judge, use `validate-evaluator`.
中文简介正在准备中。当前显示原文说明。
Write code evaluators for known failure modes with objective rules. Use when code can check the rule from a trace, with or without a reference answer. Use `write-judge-prompt` when the rule requires interpretation.
中文简介正在准备中。当前显示原文说明。
Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with unit tests per `write-code-eval`).
中文简介正在准备中。当前显示原文说明。
Build a custom browser-based annotation interface tailored to your data for reviewing LLM traces and collecting structured feedback. Use when you need to build an annotation tool, review traces, or collect human labels.
中文简介正在准备中。当前显示原文说明。
Run error analysis on a dataset. Build a review UI, select diverse samples, monitor annotations, and organize failure modes.
中文简介正在准备中。当前显示原文说明。
Run evaluations on Agent Skills to measure their effectiveness. USE FOR: "run skill evals", "evaluate my skill", "test skill quality", "check skill triggers", "skill compliance check", "measure skill performance", "run evals on [skill-name]", "grade skill execution". DO NOT USE FOR: writing skills (use skill-authoring), improving frontmatter (use sensei), or general testing unrelated to skills.
中文简介正在准备中。当前显示原文说明。
Eval-Driven Development (EDD) for AI workflows. pass@k metrics, capability evals, regression evals. Triggers: eval, edd, pass@k, capability, regression, benchmark.
中文简介正在准备中。当前显示原文说明。
Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine.
中文简介正在准备中。当前显示原文说明。
Audit an agent's context layout against the four places: system prompt, tools, history, tail. Use when the user asks to audit, review or fix their agent's context, prompt caching, token spend per turn, or CLAUDE.md / AGENTS.md layout. Read-only: reports findings and a fix list, changes nothing. Do NOT use for writing evals (evals-bootstrap) or for loop design (goal-test).
中文简介正在准备中。当前显示原文说明。
Scaffold a first eval suite for an agent: mine real failures into cases, write behavioural checks over traces, and generate the runner. Use when the user wants evals, regression tests for an agent, a golden set, or asks how to know a prompt/model change did not break things. Do NOT use for a single task's done-check (goal-test) or for auditing context (context-audit).
中文简介正在准备中。当前显示原文说明。
Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.
中文简介正在准备中。当前显示原文说明。
Use when evaluating a newly adapted model's accuracy/eval score. Runs evalscope eval (or the test/srt/run_eval.py fallback) against an OpenAI-compatible endpoint on datasets like gsm8k / aime, producing per-dataset reports and a score. Accuracy only — throughput/latency benchmarks belong to model-speed-benchmark. Triggers on accuracy, eval, evalscope, gsm8k, aime, run_eval — not throughput.
中文简介正在准备中。当前显示原文说明。
Run a skill's evals and report results. Use when the user wants to evaluate a skill, run evals, benchmark a skill, validate triggers, optimize a description, or grade skill outputs.
中文简介正在准备中。当前显示原文说明。
Braintrust is a hosted platform for evaluating and tracing LLM applications: it runs a task over a dataset, scores the outputs, stores each run as an experiment and logs production traces. Use when a user asks to write or run a Braintrust eval, score LLM outputs with autoevals, compare prompt or model versions as experiments, run evals in CI and fail on regressions, trace OpenAI or Anthropic calls with Braintrust, manage evaluation datasets, or use the bt CLI.
中文简介正在准备中。当前显示原文说明。
Use when the user wants to work on a Claude Code skill file (SKILL.md): writing one from scratch, testing whether an existing one works well, running evals or benchmarks, improving its instructions, or fixing why it isn't triggering. Triggers on: 'make a skill for X', 'test this skill', 'run evals on my SKILL.md', 'touch-skill', sharing a SKILL.md and asking if it's ready to ship. The key signal is intent to create, validate, or improve a skill — not just mention one. Do NOT trigger for general Claude Code questions, hook debugging, or CLAUDE.md configuration.
中文简介正在准备中。当前显示原文说明。
Builds QA harnesses and agent evals. Use when evaluating tool, trace, red-team, regression, multi-agent, or carried-workspace coding trajectories. Not for metric design (ai-evals).
中文简介正在准备中。当前显示原文说明。
Designs LLM, agent, responsible-AI, and multimodal evals. Use when measuring quality, fairness, grounding, safety, or judge reliability. Not for test harnesses (qa-agent-testing).
中文简介正在准备中。当前显示原文说明。
Audit existing eval coverage — gaps, metric validity, benchmark leakage, dataset freshness. Use when asked "what eval coverage do we have", "audit our evals", or "check for benchmark leakage".
中文简介正在准备中。当前显示原文说明。
Design, create, run, monitor, and maintain Vapi Simulations for assistants and squads. Use for turning plain-language requirements into a coverage plan, simulation personalities, scenarios, structured-output success criteria, simulations, suites, extending an existing suite, chat or voice runs, tool mocks, target variables, lifecycle webhooks, regression coverage, launch readiness, CI quality gates, run-result analysis, and simulation API validation errors. Do not use for fixed-turn mock-conversation Evals unless the user is deciding between Evals and Simulations.
中文简介正在准备中。当前显示原文说明。
Skill Optimization: Auto Research. Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill, benchmark skill, eval my skill, run evals on. Outputs: an improved SKILL.md, a results log, and a changelog of every mutation tried.
中文简介正在准备中。当前显示原文说明。
Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.
中文简介正在准备中。当前显示原文说明。
Use when building, migrating, or debugging Agent Evals on Inngest: scoring AI agent or workflow outcomes, deferred scorers, sessions, traces, step experiments, experiment variant attribution, Insights queries, or production eval loops for prompts, models, tools, providers, and agent behavior. Covers TypeScript SDK v4 scoring beta APIs, `scoreMiddleware`, `step.score`, `inngest.score`, `createScorer`, `defer`, `group.experiment`, `experimentRef`, `meta.sessions`, and when to use durable workflow primitives for outcome-based evaluation.
中文简介正在准备中。当前显示原文说明。