「benchmark evaluation」の検索結果

Structured SE task evaluation using 15 benchmark definitions from claude-code-harness research

日本語の概要は準備中です。原文の説明を表示しています。

Quantitative agent evaluation using 4-metric framework (correctness/step_ratio/tool_call_ratio/latency_ratio) with ideal trajectory annotation and capability-categorical taxonomy. Use when measuring agent efficiency, comparing agent variants, or gating new agents through correctness→efficiency phases. Complements harness-eval (SE benchmarks) and evaluator-optimizer (qualitative rubric).

日本語の概要は準備中です。原文の説明を表示しています。

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).

日本語の概要は準備中です。原文の説明を表示しています。

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on re...

日本語の概要は準備中です。原文の説明を表示しています。

Use when working with Kubescape — kubescape Kubernetes security posture analysis. Covers NSA/CISA framework assessment, MITRE ATT&CK mapping, CIS Kubernetes benchmarks, workload scanning, RBAC analysis, and network policy evaluation. Use when assessing Kubernetes cluster security, evaluating compliance frameworks, or analyzing workload configurations.

日本語の概要は準備中です。原文の説明を表示しています。

Use when working with Prowler — prowler cloud security assessment tool. Covers AWS, Azure, and GCP security posture assessment, CIS benchmark evaluation, compliance frameworks (PCI-DSS, HIPAA, SOC2), multi-account scanning, and remediation guidance. Use when assessing cloud security posture, running compliance audits, or investigating security findings across cloud providers.

日本語の概要は準備中です。原文の説明を表示しています。

Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.

日本語の概要は準備中です。原文の説明を表示しています。

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

日本語の概要は準備中です。原文の説明を表示しています。

Design AI evaluation frameworks with benchmarks. TRIGGERS - Use when user needs help with ai-eval-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

日本語の概要は準備中です。原文の説明を表示しています。