'evaluation' 검색 결과

採用・人材戦略の専門スキル。職務記述書（JD）作成、採用計画策定、コンピテンシーモデル設計、 面接評価基準設計、オンボーディング計画をサポート。採用要件の明確化から内定後のフォローまで 一貫した人材獲得プロセスを支援。日英両言語のテンプレートを提供し、グローバル採用にも対応。 Use when: creating job descriptions, designing competency models, planning recruitment strategies, developing interview evaluation criteria, or creating onboarding plans. Triggers: "JD作成", "採用計画", "面接評価", "オンボーディング", "コンピテンシー", "人材獲得", "job description", "recruitment plan", "interview evaluation", "hiring", "talent acquisition"

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate recommender system quality for the CSX4207 Vinyl Record Store. Use whenever you must compute or report recommender metrics (Precision@k, Recall@k, HitRate@k, MRR, MAP@k, NDCG@k, coverage, diversity, novelty, serendipity, personalization), design an evaluation protocol/split, run a baseline comparison, or write the evaluation section of a course deliverable. Covers formulas, JavaScript reference implementations, and the reporting checklist.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Rigorous benchmarking of multi-agent coordination systems: experiment design, statistical analysis, human evaluation protocols, and reproducible reporting. NOT FOR ML model evaluation (use llm-evaluation-harness), A/B testing for web products, survey design, or general data science.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Create and run automated tests that score LLM answers against expected outputs or judging criteria. Monitor progress, inspect individual results, compare models on the same tests, and use score thresholds in CI, an automated checking workflow.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating tests with expected answers
• Monitoring evaluation progress
• Inspecting failed samples and scores

Design evaluation tasks that check whether agents actually complete work. Define requests, simulated user behavior, starting conditions, and verifiable success criteria, then plan difficulty levels, training-data leakage checks, and ongoing maintenance.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating evaluation tasks for real business workflows
• Reviewing success criteria and verification logic
• Designing checks for evaluation data leakage into training

Plan and review PufferLib reinforcement learning workflows, where agents learn through rewards. Covers version-specific environment checks, parallel execution, training and evaluation plans, plus checkpoint inspection before loading saved models.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose APIs for a specific PufferLib version
• Check environment behavior before training
• Create training and evaluation plans with separate seeds

Prepare and check research documents for clinical decision-support evaluation, including evidence profiles, cohort tables, survival plans, and model assessments. Work with aggregate or synthetic inputs, record sources and review status, and keep outputs outside patient care.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing research model evaluation documents
• Checking evidence profiles and source links
• Reviewing survival analysis plans

Implement .NET workflows for Azure AI Foundry projects, including agent creation, resource connections, dataset uploads, model deployment inspection, search index management, and evaluations. Examples also cover authentication, asynchronous operations, and resource cleanup.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating and running agents from a .NET application
• Inspecting resource connections and model deployments
• Uploading datasets and managing search indexes

Implement Java workflows for Azure AI Foundry projects using SDK examples for authentication, connected resources, datasets, search indexes, and model evaluations. The skill also covers client setup, paginated resource lists, and error handling.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• List connected Azure resources
• Create or update search indexes
• Implement access to model evaluations

Compare prompt revisions on a fixed set of test cases to identify regressions, meaning behavior that gets worse after a change. Record model settings, case-level evidence, missing results, costs, and evaluation limits for a reviewable decision.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing original and revised prompts
• Checking required JSON fields after prompt changes
• Reviewing tool selection and task completion

Design criteria for using an LLM to judge AI responses through scoring or comparisons between two answers. Includes checks for order and length bias, plus validation against human judgments to assess evaluation reliability.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build an automated evaluation pipeline for AI responses
• Compare responses after changing a prompt or model
• Define shared scoring criteria for reviewers

Explains how PluginEval combines structural checks and experimental model assessments into quality scores. Helps interpret low scores, identify improvement areas, set marketplace thresholds, and explain what evaluation badges actually represent.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating why a skill received a low score
• Improving descriptions that define when a skill should run
• Setting score thresholds for marketplace listings

Prepare an evaluation framework before fine-tuning a language model. Build labeled examples, checks for each failure type, and a base-model benchmark, while calibrating model-based judges against human labels to support comparisons after training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build an evaluation set from agent traces
• Create separate checks for each failure type
• Calibrate an LLM judge against human labels

Turn graded execution traces and production logs into model training examples and preferred-versus-rejected response pairs. Select useful records, remove sensitive information, exclude evaluation examples, and preserve links to the original runs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Convert high-scoring successful runs into training examples
• Use expert-corrected failures as training examples
• Pair passing and failing responses to the same task

Build an end-to-end evaluation pipeline for Python AI applications using pixie-qa and real language-model calls. Define criteria, prepare realistic test data, score application runs, and analyze the evidence to produce a prioritized improvement plan.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up quality checks for Python AI applications
• Creating realistic scenarios for known failure modes
• Scoring full application runs with real language-model calls

Create and manage Arize datasets for evaluation and experiments using the ax CLI. Import records from files, append examples, inspect dataset versions, and export a chosen version for offline analysis, with checks for field consistency and export completeness.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating evaluation datasets from files
• Adding examples to an existing test set
• Inspecting dataset metadata and versions

Design workflows that evaluate and refine AI outputs through self-review, separate generation and evaluation roles, tests, or weighted scoring criteria. Includes patterns for structured feedback, iteration limits, and stopping when quality stops improving.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Adding self-review and revision loops to AI outputs
• Separating generation from evaluation
• Refining generated code using test results

Plan technology strategy and engineering team growth with frameworks for technical debt, hiring, architecture decisions, and vendor evaluation. Use Python tools and decision templates to prioritize improvements, structure teams, and track engineering performance.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Prioritizing technical debt reduction
• Planning hiring and engineering team structure
• Comparing technologies and vendors