'evaluation' 검색 결과

Plan technology strategy and engineering team growth with frameworks for technical debt, hiring, architecture decisions, and vendor evaluation. Use Python tools and decision templates to prioritize improvements, structure teams, and track engineering performance.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Prioritizing technical debt reduction
• Planning hiring and engineering team structure
• Comparing technologies and vendors

Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models and prompts
• Checking quality before deployment
• Evaluating search quality in RAG applications

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Supports preparation for EU medical device regulation reviews by organizing device classification, technical documentation, clinical evidence, and post-market monitoring. Includes a tool to identify documentation and compliance gaps and prioritize the next steps.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Documenting a device classification rationale under MDR
• Checking technical documentation before a review
• Planning clinical evaluation and identifying evidence gaps

Evaluate Google Cloud workloads against built-in best practices or custom organizational rules using Workload Manager. Use public client libraries or the REST API to run checks, review violations and scanned resources, and export results to BigQuery.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Evaluate workloads against Google Cloud best practices
• Inspect rules for SAP and SQL Server workloads
• Run evaluations using custom organizational rules

Evaluate AI models and agents on Google Cloud using test datasets and scoring criteria. Generate simulated user scenarios, analyze failures, and compare results across changes to guide improvements to prompts and code.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build evaluation datasets from conversation logs
• Generate simulated user scenarios
• Choose scoring criteria for responses and tool use

Guides machine learning work from data preparation and model training to evaluation and production monitoring. Covers PyTorch, scikit-learn, experiment tracking, and reproducible workflows so teams can compare models and maintain them after deployment.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build a baseline model for comparison
• Review data preparation for evaluation leakage
• Track experiments and reproduce training runs

Guides iterative improvement of AI agent prompts, skills, and memory using A-Evolve. It explains how to evaluate agents against benchmark tasks, apply changes, validate results, and track or roll back revisions with Git.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Improve an existing agent against benchmark tasks
• Build evaluations for domain-specific tasks
• Refine skills and memory from failure feedback

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Design and improve prompts for language models, then test their accuracy and consistency. Create reusable templates, JSON output schemas that define response structure, evaluation criteria, and test cases while considering token use and model differences.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing prompts for a new AI application
• Improving inconsistent model responses
• Defining JSON response formats

Build evaluation suites for AI features with the Axiom AI SDK. Generate test files, scoring functions, test cases, and configurable parameters from descriptions and existing code to compare changes and monitor output quality in production.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Add regression evaluations for an AI feature
• Score classification, retrieval, and structured outputs
• Compare model and parameter configurations

Execute an existing Ouroboros Seed, a specification containing a goal and acceptance criteria, as a background job. The workflow checks each criterion, normally queues formal evaluation, and follows a bounded improvement loop if evaluation rejects the result.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Executing an existing Seed specification
• Checking whether each acceptance criterion is met
• Following execution, evaluation, and improvement results

Evaluate AI agents by preparing test cases, scoring responses and tool use, investigating failures, and comparing results after changes. Supports built-in checks and custom scoring rules, including evaluations performed by a language model.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Create test cases for an AI agent
• Check response quality and tool use
• Investigate failed evaluation cases

Create Vally evaluation specifications for skills, custom agents, and workflow packages. Design distinct test requests, prepare fixture files, define automated checks and outcome rubrics, and size comparisons to provide meaningful evidence without favoring the skill’s wording.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Create an eval.yaml for a skill or custom agent
• Define outcome-based checks and judging rubrics
• Prepare fixture files that reproduce specific failures

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design evaluation datasets and supporting documentation for AI systems and agents, covering the scoping interview, difficulty distribution, dataset contract, sample review, and metric and tooling selection. Use when building or reviewing an evaluation set for a conversational agent, assistant, or retrieval-grounded AI system.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when evaluating the model and analyzing results for an Agricultural Systems (AgSy) manuscript so it survives expert systems review — independent model evaluation (observed vs. simulated, fit statistics), sensitivity and uncertainty analysis, and trade-off / scenario analysis across the system. Guides evaluation norms; it does not fabricate results or run the model.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate, select, and contract with vendors and SaaS tools. Use this skill when comparing alternatives, running an RFP, scoring vendors against criteria, negotiating contracts, planning a switch, or assessing a vendor's risk. Triggers on vendor evaluation, RFP, vendor selection, build vs buy, SaaS evaluation, vendor scorecard, vendor comparison, contract negotiation, vendor switch, procurement. Also triggers when a renewal is coming up or when a tool isn't meeting expectations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design a source evaluation protocol using lateral reading and credibility checks for digital information. Use when students need to evaluate websites, online sources, or social media claims.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Provides weighted scoring, rubrics, and decision-threshold patterns. Use when designing quality gates, evaluation systems, or decision frameworks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.