“pass@k”的搜索结果

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Evaluate code generation models with HumanEval, MBPP, and multilingual benchmarks. The skill guides test execution and model comparisons using pass@k, an estimate of whether at least one of k generated solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Compare code generation models on shared benchmarks
• Measure code generation performance across languages
• Evaluate instruction-tuned coding models

Evaluate AI agent outputs with code checks and model-based grading. Run repeated trials to measure reliability, compare prompts or models, and check whether changes reduce quality, with saved transcripts for reviewing the results.

中文简介正在准备中。当前显示英语简介。

• Comparing two prompts on the same test cases
• Checking quality after agent configuration changes
• Turning real failures into repeatable evaluation cases

在AI开发前定义成功条件，使用代码、规则、模型或人工评估新功能并检查既有功能是否正常，再记录多次尝试的可靠性指标与报告，用于比较提示词、代理或模型版本的变化。

• 在AI实现前确定成功条件
• 检查提示词或代理变更后的回归问题
• 通过多次尝试衡量可靠性

Plan further training for AI agent models by diagnosing gaps in knowledge, output consistency, or decision making. Compare continued pretraining, supervised fine-tuning, reinforcement learning, and distillation, then design training data, feedback environments, and evaluation checks.

中文简介正在准备中。当前显示英语简介。

• Choosing a training approach for an agent model
• Designing training examples and source data
• Planning reward checks and reinforcement learning environments

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

中文简介正在准备中。当前显示原文说明。

Eval-Driven Development (EDD) for AI workflows. pass@k metrics, capability evals, regression evals. Triggers: eval, edd, pass@k, capability, regression, benchmark.

中文简介正在准备中。当前显示原文说明。

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

中文简介正在准备中。当前显示原文说明。

Avalia modelos de geração de código em HumanEval, MBPP, MultiPL-E e 15+ benchmarks com métricas pass@k. Use ao comparar modelos de código, avaliar capacidades de codificação, testar suporte multilíngue ou medir qualidade de geração de código. Padrão da indústria do BigCode Project usado pelos leaderboards do HuggingFace.

中文简介正在准备中。当前显示原文说明。

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

中文简介正在准备中。当前显示原文说明。

Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.

中文简介正在准备中。当前显示英语简介。

• Building an initial agent evaluation process
• Comparing models on actual work tasks
• Investigating a drop in evaluation scores