'agent-eval' 검색 결과

Benchmark how CodeGraph affects an agent investigating a real codebase. Choose a version, language, repository, and execution mode, then compare cost, tool calls, file reads, and answer correctness with and without CodeGraph.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Evaluating a local development build
• Auditing a published CodeGraph version
• Comparing investigation cost and file reads

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Review board (Busabase App-in-Skill) that runs a fixed suite of mock test cases against a baseline vs candidate agent version and surfaces rubric-scored regressions before a release. Use when the user invokes $kelly-agent-eval or /kelly-agent-eval, wants to review agent-version regressions, compare baseline vs candidate quality, triage a release, or record a release approve/block decision. Deterministic mock rubric scores only — not a real LLM-judge call, and it never deploys anything.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Separating autonomous and human-assisted success rates
• Reporting evaluations with retries and invalid runs
• Comparing agents or releases under matched conditions

[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics. Use when the user says: agent-evaluation, agent evaluation, agent eval, agent benchmark, executor evaluation, executor benchmark, compare agents, compare codex claude.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Expert en évaluation d'agents IA (benchmarking, human eval, latence, coût, précision, safety testing)

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

실제 개발 과제로 여러 코딩 에이전트를 비교합니다. 성공률, 확인 가능한 API 비용, 소요 시간과 반복 실행의 안정성을 측정해 도입 결정과 업데이트 후 평가에 활용합니다.

• 실제 개발 과제로 에이전트를 비교합니다
• 새 도구나 모델을 도입하기 전에 평가합니다
• 에이전트 업데이트 후 성능을 확인합니다

Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Building an initial agent evaluation process
• Comparing models on actual work tasks
• Investigating a drop in evaluation scores

Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Recording agent evaluation results in a shared format
• Comparing a candidate agent with a baseline
• Tracking actual cost per solved task

Build repeatable evaluations for AI agent responses, tool choices, and multi-step tasks. Use reference datasets and scoring criteria to compare changes, detect quality regressions, and set automated checks that must pass before deployment.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare agent quality before and after prompt changes
• Test tool selection and multi-step task execution
• Evaluate resistance to prompt injection

Compare agent, prompt, or tool configuration changes using versioned test cases and explicit outcome checks. Record each attempt, distinguish agent failures from evaluation or infrastructure problems, and report regressions, permission violations, and uncertainty.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare agent behavior before and after prompt changes
• Reproduce an observed failure with preserved evidence
• Run regression tests after a bounded fix

Design tests that assess an AI agent's capabilities and reliability. Covers repeated runs to measure variation, checks for required behavior, and adversarial testing to expose failures that ordinary successful examples may miss.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Measuring variation across repeated runs of the same test
• Checking whether an agent follows required behavior
• Testing responses to unexpected or adversarial inputs

Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate and improve Claude Code commands, skills, and agents. Use when testing prompt effectiveness, validating context engineering choices, or measuring improvement quality.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use to test whether an AI agent, prompt, or skill actually works before or after it ships — build a small eval set, score runs, and catch regressions when the prompt or model changes. Trigger when the user asks whether a prompt is good, wants to compare models or versions, sees inconsistent agent output, or is about to put an agent in front of customers.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.