70 skills · sorted by relevance
Benchmark how CodeGraph affects an agent investigating a real codebase. Choose a version, language, repository, and execution mode, then compare cost, tool calls, file reads, and answer correctness with and without CodeGraph.
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Review board (Busabase App-in-Skill) that runs a fixed suite of mock test cases against a baseline vs candidate agent version and surfaces rubric-scored regressions before a release. Use when the user invokes $kelly-agent-eval or /kelly-agent-eval, wants to review agent-version regressions, compare baseline vs candidate quality, triage a release, or record a release approve/block decision. Deterministic mock rubric scores only — not a real LLM-judge call, and it never deploys anything.
The English summary is being prepared. Showing the original description.
Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.
[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics. Use when the user says: agent-evaluation, agent evaluation, agent eval, agent benchmark, executor evaluation, executor benchmark, compare agents, compare codex claude.
The English summary is being prepared. Showing the original description.
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
The English summary is being prepared. Showing the original description.
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
The English summary is being prepared. Showing the original description.
Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.
The English summary is being prepared. Showing the original description.
Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo.
The English summary is being prepared. Showing the original description.
Expert en évaluation d'agents IA (benchmarking, human eval, latence, coût, précision, safety testing)
The English summary is being prepared. Showing the original description.
Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.
The English summary is being prepared. Showing the original description.
Compare coding agents on tasks from your own codebase. Define success checks, run agents in separate Git working directories, and report pass rates, available API costs, completion time, and consistency across repeated trials.
Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.
Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.
Build repeatable evaluations for AI agent responses, tool choices, and multi-step tasks. Use reference datasets and scoring criteria to compare changes, detect quality regressions, and set automated checks that must pass before deployment.
Compare agent, prompt, or tool configuration changes using versioned test cases and explicit outcome checks. Record each attempt, distinguish agent failures from evaluation or infrastructure problems, and report regressions, permission violations, and uncertainty.
Design tests that assess an AI agent's capabilities and reliability. Covers repeated runs to measure variation, checks for required behavior, and adversarial testing to expose failures that ordinary successful examples may miss.
Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".
The English summary is being prepared. Showing the original description.
Evaluate and improve Claude Code commands, skills, and agents. Use when testing prompt effectiveness, validating context engineering choices, or measuring improvement quality.
The English summary is being prepared. Showing the original description.
Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.
The English summary is being prepared. Showing the original description.
Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.
The English summary is being prepared. Showing the original description.
Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.
The English summary is being prepared. Showing the original description.
Use to test whether an AI agent, prompt, or skill actually works before or after it ships — build a small eval set, score runs, and catch regressions when the prompt or model changes. Trigger when the user asks whether a prompt is good, wants to compare models or versions, sees inconsistent agent output, or is about to put an agent in front of customers.
The English summary is being prepared. Showing the original description.