41 skills · sorted by relevance
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.
[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics. Use when the user says: agent-evaluation, agent evaluation, agent eval, agent benchmark, executor evaluation, executor benchmark, compare agents, compare codex claude.
The English summary is being prepared. Showing the original description.
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
The English summary is being prepared. Showing the original description.
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.
The English summary is being prepared. Showing the original description.
Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.
The English summary is being prepared. Showing the original description.
Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.
Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.
Compare agent, prompt, or tool configuration changes using versioned test cases and explicit outcome checks. Record each attempt, distinguish agent failures from evaluation or infrastructure problems, and report regressions, permission violations, and uncertainty.
Design tests that assess an AI agent's capabilities and reliability. Covers repeated runs to measure variation, checks for required behavior, and adversarial testing to expose failures that ordinary successful examples may miss.
Evaluate and improve Claude Code commands, skills, and agents. Use when testing prompt effectiveness, validating context engineering choices, or measuring improvement quality.
The English summary is being prepared. Showing the original description.
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
The English summary is being prepared. Showing the original description.
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
The English summary is being prepared. Showing the original description.
Design reproducible evaluations for AI agents with representative task sets, explicit rubrics, appropriate graders, baselines, regression gates, and failure analysis. Use when defining agent quality, comparing prompts or models, validating a release, measuring tool-use reliability, investigating regressions, or deciding whether an agent is ready for production.
The English summary is being prepared. Showing the original description.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarksUse when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" mentioned.
The English summary is being prepared. Showing the original description.
Evaluate agent and skill behavior or routing.
The English summary is being prepared. Showing the original description.
Designs and runs reproducible evaluations for AI agents, prompts, tools, skills, and model-backed workflows using realistic datasets, isolated baselines, objective assertions, rubric grading, trajectory analysis, cost/latency tracking, and regression comparison. Use when measuring agent quality, optimizing skill triggering, comparing prompts or models, or gating an AI feature release. Not for ordinary deterministic unit tests.
The English summary is being prepared. Showing the original description.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
The English summary is being prepared. Showing the original description.
LLM-as-judge evaluation framework with 5-dimension rubric (accuracy, groundedness, coherence, completeness, helpfulness) for scoring AI-generated content quality with weighted composite scores and evidence citations
The English summary is being prepared. Showing the original description.
Evaluate AI agents rigorously — task benchmarks, success criteria, failure taxonomy, cost/latency tracking, and regression testing. Use when you need to know whether an agent actually works, not whether it demos well.
The English summary is being prepared. Showing the original description.
Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures.
The English summary is being prepared. Showing the original description.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
The English summary is being prepared. Showing the original description.