'pass@k' 검색 결과

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Evaluate code generation models with HumanEval, MBPP, and multilingual benchmarks. The skill guides test execution and model comparisons using pass@k, an estimate of whether at least one of k generated solutions passes the tests.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare code generation models on shared benchmarks
• Measure code generation performance across languages
• Evaluate instruction-tuned coding models

Evaluate AI agent outputs with code checks and model-based grading. Run repeated trials to measure reliability, compare prompts or models, and check whether changes reduce quality, with saved transcripts for reviewing the results.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing two prompts on the same test cases
• Checking quality after agent configuration changes
• Turning real failures into repeatable evaluation cases

AI 개발 전에 성공 조건을 정하고 코드·규칙·모델·사람의 평가로 새 기능과 기존 기능을 확인합니다. 여러 차례 시도한 결과에서 신뢰성을 기록합니다.

• AI 구현 전에 성공 조건 정의
• 프롬프트나 에이전트 변경 후 회귀 확인
• 여러 차례 시도로 신뢰성 측정

Plan further training for AI agent models by diagnosing gaps in knowledge, output consistency, or decision making. Compare continued pretraining, supervised fine-tuning, reinforcement learning, and distillation, then design training data, feedback environments, and evaluation checks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choosing a training approach for an agent model
• Designing training examples and source data
• Planning reward checks and reinforcement learning environments

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Eval-Driven Development (EDD) for AI workflows. pass@k metrics, capability evals, regression evals. Triggers: eval, edd, pass@k, capability, regression, benchmark.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Avalia modelos de geração de código em HumanEval, MBPP, MultiPL-E e 15+ benchmarks com métricas pass@k. Use ao comparar modelos de código, avaliar capacidades de codificação, testar suporte multilíngue ou medir qualidade de geração de código. Padrão da indústria do BigCode Project usado pelos leaderboards do HuggingFace.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Building an initial agent evaluation process
• Comparing models on actual work tasks
• Investigating a drop in evaluation scores