70건 · 관련도순
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Reviews the instructions text and skills of a declarative agent, with its stated capabilities, knowledge sources and audience, for prompt-injection exposure, data-leakage paths, over-broad permissions and missing refusals, and returns an attack-surface map, a findings table with severity, quoted evidence and attack path, one pasteable fix per finding, a probe pack the owner can run and a refusal-coverage table. Use when the user asks to "red-team this agent", "check these instructions for prompt injection", "could this agent leak data", "review the permissions on my agent" or "what should this agent refuse". Do not use for checking a SKILL.md against the format rules, use skill-file-reviewer instead; do not use for designing the launch test set, use agent-evaluation-plan instead. Drafts for human review; never approves, authorises or signs off.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Designs a pre-launch evaluation plan for a declarative agent from its instructions text, skills, capabilities, knowledge sources and sample requests: a behaviour contract of testable statements quoted from the text, test prompts by scenario family and tier with setup, expected behaviour, pass and fail criteria and red flags, a red-flag catalogue, a blank scoring sheet, a coverage table and a go or no-go checklist the owner completes and decides. Use when the user asks to "write test cases for my agent", "how do I evaluate this agent before launch", "build a test plan for the agent", "what should I check before publishing this agent" or "give me a go no-go checklist". Do not use for finding injection or leakage exposures in the instructions, use agent-instructions-red-team instead; do not use for reviewing one SKILL.md, use skill-file-reviewer instead. Drafts for human review; never approves, authorises or signs off.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Reviews one SKILL.md against the format rules and returns a findings table: strict YAML front matter with exactly name and description, name equal to the folder, description as the trigger, the eight-heading skeleton, capability-neutral wording, safety boundaries. Each finding carries severity, rule identifier, location, quoted evidence and replacement text; the verdict READY, REVISE or BLOCKED is an apparent state and the owner decides. Use when the user asks to "review this SKILL.md", "check my skill file before upload", "lint this skill description", "validate the front matter of this skill" or "score this skill folder". Do not use to write an agent's instructions field, use agent-instructions-drafter instead; for injection or leakage exposures use agent-instructions-red-team; for launch tests use agent-evaluation-plan. Drafts for human review; never approves, authorises or signs off.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Build command-only adapters for `eval` run mode using strict stdin/stdout JSON contracts.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use test-driven development for behavior-changing feature or fix work, and whenever the user mentions TDD, test-first, red-green-refactor, tracer bullets, integration tests, or public-interface behavior tests. Skip for docs-only, path-only rename, formatting-only, or purely mechanical chores unless explicitly requested.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
End-to-end eval-suite orchestration with the `eval` command: run -> grade -> compare -> calibrate, using strict JSON contracts and schema discovery.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Analyze graded `trial_result` JSONL outputs from the eval pipeline, including reliability and custom slicing.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on re...
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
コーディングエージェント（Claude Code、Aider、Codexなど）のカスタムタスクによる直接比較。合格率、コスト、時間、一貫性メトリクスを測定
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.