“evaluation”的搜索结果

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

中文简介正在准备中。当前显示英语简介。

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Evaluate language models with over 100 benchmark tests covering knowledge, math, coding, safety, and image understanding. Configure runs on Docker, Slurm computing clusters, or cloud infrastructure, compare models, and export results.

中文简介正在准备中。当前显示英语简介。

• Compare models using the same benchmark tests
• Run large evaluations on a Slurm cluster
• Evaluate safety and image understanding

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Design evaluation systems that measure agent results across accuracy, completeness, citations, and tool efficiency. Build repeatable tests, compare changes against a baseline, and define quality checks that catch regressions before deployment and monitor production performance.

中文简介正在准备中。当前显示英语简介。

• Compare agent configurations against a baseline
• Catch quality regressions before deployment
• Measure the effects of context strategies

Evaluate a driving agent on CARLA Leaderboard routes after checking the required setup. Select routes and repetitions, save driving scores and infractions, record runs, and resume interrupted evaluations from a saved checkpoint.

中文简介正在准备中。当前显示英语简介。

• Smoke-test a Leaderboard setup on one route
• Evaluate an agent across validation routes
• Resume an evaluation after a crash

Evaluate NVIDIA Cosmos Policy for robot manipulation in LIBERO and RoboCasa simulations. Follow GPU environment setup, configure rendering without a display, run small checks before larger benchmarks, inspect success rates, and profile inference latency.

中文简介正在准备中。当前显示英语简介。

• Running a small LIBERO evaluation
• Evaluating kitchen tasks in RoboCasa
• Setting up simulation on a GPU cluster without a display

Evaluate language models on knowledge, math, coding, safety, and image understanding benchmarks. The skill guides reproducible runs in containers using local Docker, Slurm clusters, or cloud execution, with shared configurations for model comparison and result export.

中文简介正在准备中。当前显示英语简介。

• Comparing models on the same benchmark tasks
• Running a small evaluation sample before a full benchmark
• Running large evaluations on a Slurm cluster

Evaluate code generation models with HumanEval, MBPP, and multilingual benchmarks. The skill guides test execution and model comparisons using pass@k, an estimate of whether at least one of k generated solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Compare code generation models on shared benchmarks
• Measure code generation performance across languages
• Evaluate instruction-tuned coding models

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

中文简介正在准备中。当前显示英语简介。

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Evaluate Hugging Face Hub models using inspect-ai or lighteval, with local inference or a provider-backed option. Choose an inference engine, start with a small test, and switch engines when model compatibility requires it.

中文简介正在准备中。当前显示英语简介。

• Run benchmark evaluations on a local GPU
• Choose an evaluation framework and inference engine
• Check a small sample before a full evaluation

[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics. Use when the user says: agent-evaluation, agent evaluation, agent eval, agent benchmark, executor evaluation, executor benchmark, compare agents, compare codex claude.

中文简介正在准备中。当前显示原文说明。

Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison".

中文简介正在准备中。当前显示原文说明。

Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.

中文简介正在准备中。当前显示原文说明。

Use when the user has run multiple evaluation skills and wants a comprehensive analysis — maturity assessment, cross-skill signals, trends, prioritized actions, and an executive summary. Also use when the user mentions eval health check, evaluation audit, ship readiness, evaluation maturity, or "how good is my evaluation system itself." This is a read-only analysis skill.

中文简介正在准备中。当前显示原文说明。

Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.

中文简介正在准备中。当前显示原文说明。

Create and check long-running video material evaluation tasks. Use this skill when the user wants to submit videos for evaluation, check an existing video evaluation task list, or fetch the result of a previously created video evaluation task. This skill is not a general-purpose video upload skill because upload is allowed only as an internal step of task creation. Authentication uses an API key passed as an Authorization bearer token.

中文简介正在准备中。当前显示原文说明。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

中文简介正在准备中。当前显示原文说明。

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

中文简介正在准备中。当前显示原文说明。

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

中文简介正在准备中。当前显示原文说明。

Use when: AI teammate / AIチームメイト / AI チームメイト / Agent Template / Foundry Autopilot / Agent 365 / 同僚エージェントの開発・公開。既定は Foundry hosted agent を Agent template として発行する経路。publish_foundry_autopilot.py が必須で、汎用 scripts/deploy.py や SDK agents.create_version だけで完了にしない。明示的な Agent Template / Autopilot 要求の前提が不足したら停止し、通常 Agent・自己ホストへの自動フォールバックは禁止。明示選択の自己ホストは別経路として対応する。M365 Requests の Type=Agent template と Teams Agents for your team での hire を手動確認するまで未完了。初回 scaffold で評価Hub（Code Apps）・Python EvaluationWorker・回帰テスト、publish 時に Foundry Monitor/Evaluations を構成する。CI/CD・レビューゲートは alm に委譲する。

中文简介正在准备中。当前显示原文说明。

Use when testing, evaluating, or building regression suites for Agentforce agents: conversation testing in Agent Builder, topic (now subagent) coverage and utterance testing, Testing API and AiEvaluationDefinition metadata, Agentforce DX CLI test runs (sf agent generate test-spec, sf agent test create/run/resume/results/list), evaluation metrics (containment rate, escalation rate, CSAT, topic activation accuracy), and post-deploy analytics via Enhanced Event Logs. Triggers: 'how do I test my Agentforce agent', 'agent routes to wrong subagent', 'write utterance tests', 'regression test after topic change', 'measure agent quality', 'agent containment rate', 'run agent tests from the CLI'. NOT for agent creation, topic design, or action contract design — use agentforce/agentforce-agent-creation, agentforce/agent-topic-design, or agentforce/agent-actions respectively.

中文简介正在准备中。当前显示原文说明。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

中文简介正在准备中。当前显示原文说明。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

中文简介正在准备中。当前显示原文说明。

採用・人材戦略の専門スキル。職務記述書（JD）作成、採用計画策定、コンピテンシーモデル設計、 面接評価基準設計、オンボーディング計画をサポート。採用要件の明確化から内定後のフォローまで 一貫した人材獲得プロセスを支援。日英両言語のテンプレートを提供し、グローバル採用にも対応。 Use when: creating job descriptions, designing competency models, planning recruitment strategies, developing interview evaluation criteria, or creating onboarding plans. Triggers: "JD作成", "採用計画", "面接評価", "オンボーディング", "コンピテンシー", "人材獲得", "job description", "recruitment plan", "interview evaluation", "hiring", "talent acquisition"

中文简介正在准备中。当前显示原文说明。