'evaluation' 검색 결과

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Set up Langfuse to record LLM application activity, costs, and response times. Manage prompt versions, organize evaluation datasets, and compare results to investigate failures and guide improvements using recorded evidence.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Comparing prompt versions and evaluation scores
• Monitoring LLM costs and response times

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Design evaluation systems that measure agent results across accuracy, completeness, citations, and tool efficiency. Build repeatable tests, compare changes against a baseline, and define quality checks that catch regressions before deployment and monitor production performance.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare agent configurations against a baseline
• Catch quality regressions before deployment
• Measure the effects of context strategies

Evaluate a driving agent on CARLA Leaderboard routes after checking the required setup. Select routes and repetitions, save driving scores and infractions, record runs, and resume interrupted evaluations from a saved checkpoint.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Smoke-test a Leaderboard setup on one route
• Evaluate an agent across validation routes
• Resume an evaluation after a crash

Evaluate NVIDIA Cosmos Policy for robot manipulation in LIBERO and RoboCasa simulations. Follow GPU environment setup, configure rendering without a display, run small checks before larger benchmarks, inspect success rates, and profile inference latency.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a small LIBERO evaluation
• Evaluating kitchen tasks in RoboCasa
• Setting up simulation on a GPU cluster without a display

Evaluate code generation models with HumanEval, MBPP, and multilingual benchmarks. The skill guides test execution and model comparisons using pass@k, an estimate of whether at least one of k generated solutions passes the tests.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare code generation models on shared benchmarks
• Measure code generation performance across languages
• Evaluate instruction-tuned coding models

Evaluate language models on knowledge, math, coding, safety, and image understanding benchmarks. The skill guides reproducible runs in containers using local Docker, Slurm clusters, or cloud execution, with shared configurations for model comparison and result export.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models on the same benchmark tasks
• Running a small evaluation sample before a full benchmark
• Running large evaluations on a Slurm cluster

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Evaluate Hugging Face Hub models using inspect-ai or lighteval, with local inference or a provider-backed option. Choose an inference engine, start with a small test, and switch engines when model compatibility requires it.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Run benchmark evaluations on a local GPU
• Choose an evaluation framework and inference engine
• Check a small sample before a full evaluation

[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics. Use when the user says: agent-evaluation, agent evaluation, agent eval, agent benchmark, executor evaluation, executor benchmark, compare agents, compare codex claude.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when the user has run multiple evaluation skills and wants a comprehensive analysis — maturity assessment, cross-skill signals, trends, prioritized actions, and an executive summary. Also use when the user mentions eval health check, evaluation audit, ship readiness, evaluation maturity, or "how good is my evaluation system itself." This is a read-only analysis skill.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Create and check long-running video material evaluation tasks. Use this skill when the user wants to submit videos for evaluation, check an existing video evaluation task list, or fetch the result of a previously created video evaluation task. This skill is not a general-purpose video upload skill because upload is allowed only as an internal step of task creation. Authentication uses an API key passed as an Authorization bearer token.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when: AI teammate / AIチームメイト / AI チームメイト / Agent Template / Foundry Autopilot / Agent 365 / 同僚エージェントの開発・公開。既定は Foundry hosted agent を Agent template として発行する経路。publish_foundry_autopilot.py が必須で、汎用 scripts/deploy.py や SDK agents.create_version だけで完了にしない。明示的な Agent Template / Autopilot 要求の前提が不足したら停止し、通常 Agent・自己ホストへの自動フォールバックは禁止。明示選択の自己ホストは別経路として対応する。M365 Requests の Type=Agent template と Teams Agents for your team での hire を手動確認するまで未完了。初回 scaffold で評価Hub（Code Apps）・Python EvaluationWorker・回帰テスト、publish 時に Foundry Monitor/Evaluations を構成する。CI/CD・レビューゲートは alm に委譲する。

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when testing, evaluating, or building regression suites for Agentforce agents: conversation testing in Agent Builder, topic (now subagent) coverage and utterance testing, Testing API and AiEvaluationDefinition metadata, Agentforce DX CLI test runs (sf agent generate test-spec, sf agent test create/run/resume/results/list), evaluation metrics (containment rate, escalation rate, CSAT, topic activation accuracy), and post-deploy analytics via Enhanced Event Logs. Triggers: 'how do I test my Agentforce agent', 'agent routes to wrong subagent', 'write utterance tests', 'regression test after topic change', 'measure agent quality', 'agent containment rate', 'run agent tests from the CLI'. NOT for agent creation, topic design, or action contract design — use agentforce/agentforce-agent-creation, agentforce/agent-topic-design, or agentforce/agent-actions respectively.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

採用・人材戦略の専門スキル。職務記述書（JD）作成、採用計画策定、コンピテンシーモデル設計、 面接評価基準設計、オンボーディング計画をサポート。採用要件の明確化から内定後のフォローまで 一貫した人材獲得プロセスを支援。日英両言語のテンプレートを提供し、グローバル採用にも対応。 Use when: creating job descriptions, designing competency models, planning recruitment strategies, developing interview evaluation criteria, or creating onboarding plans. Triggers: "JD作成", "採用計画", "面接評価", "オンボーディング", "コンピテンシー", "人材獲得", "job description", "recruitment plan", "interview evaluation", "hiring", "talent acquisition"

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.