'eval' 검색 결과

Create and run automated tests that score LLM answers against expected outputs or judging criteria. Monitor progress, inspect individual results, compare models on the same tests, and use score thresholds in CI, an automated checking workflow.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating tests with expected answers
• Monitoring evaluation progress
• Inspecting failed samples and scores

대규모 언어 모델의 답변을 채점하는 평가 세트를 만들고 실행합니다. 진행 상황과 개별 답변의 통과 여부를 확인하고, 모델 비교나 CI의 품질 판정에도 활용합니다.

• 입력과 예상 답변으로 평가를 만듭니다
• 평가 진행 상황을 확인합니다
• 실패한 답변과 점수를 조사합니다

기존 테스트에서 고정된 JSON 평가 데이터를 만들고 형식 오류를 검사합니다. 커밋이나 파생 버전이 달라도 같은 과제를 사용해 동일한 기준으로 개선안을 비교합니다.

• 기존 테스트에서 고정 평가 세트를 만듭니다
• CI에서 평가 파일을 검증합니다
• 같은 과제로 여러 커밋을 비교합니다

Benchmark how CodeGraph affects an agent investigating a real codebase. Choose a version, language, repository, and execution mode, then compare cost, tool calls, file reads, and answer correctness with and without CodeGraph.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Evaluating a local development build
• Auditing a published CodeGraph version
• Comparing investigation cost and file reads

Build and evaluate time-to-event models with scikit-survival, including records whose events remain unobserved when follow-up ends. Keep preprocessing separate from evaluation data, compare models, predict survival probabilities, and summarize censoring-aware metrics.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Validate event indicators and follow-up times in local CSV data
• Train survival models without leaking evaluation data
• Evaluate survival probabilities with censoring-aware metrics

Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Recording agent evaluation results in a shared format
• Comparing a candidate agent with a baseline
• Tracking actual cost per solved task

Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Separating autonomous and human-assisted success rates
• Reporting evaluations with retries and invalid runs
• Comparing agents or releases under matched conditions

Prepare an evaluation framework before fine-tuning a language model. Build labeled examples, checks for each failure type, and a base-model benchmark, while calibrating model-based judges against human labels to support comparisons after training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build an evaluation set from agent traces
• Create separate checks for each failure type
• Calibrate an LLM judge against human labels

Evaluate LLM applications using automated scores, human reviews, and model-based judging. Build repeatable checks to compare models or prompts, assess answer quality, and detect declines in performance before deployment.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models and prompts
• Checking quality after prompt changes
• Detecting performance declines before deployment

Run repeated change-and-evaluation cycles for one predefined improvement mission within an explicit runtime limit. Save each evaluation as JSON and each decision in readable Markdown logs, with support for periodic reruns through Claude Code native cron.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Repeating improvements under a fixed evaluation method
• Keeping evaluation results and decision logs
• Running experiments within a time limit

Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models and prompts
• Checking quality before deployment
• Evaluating search quality in RAG applications

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Turn a feature into an agent skill or improve an existing one. Define purpose-specific quality criteria, review outputs with models from three providers, then add tests and check that user requests reach the intended skill.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Turn a reusable feature into an agent skill
• Audit an existing skill for missing parts
• Check quality after fixing or extending a skill

Improve prompts through test cases and baseline comparisons, measure how well AI answers use retrieved sources, and check agent configurations. Includes Python tools for prompt analysis, retrieval quality reports, and token estimates, with costs calculated from supplied rates.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare prompt revisions against a baseline
• Measure answer quality against retrieved sources
• Check agent tool settings and loop risks

Evaluate a Claude Managed Agent against a written rubric, a set of checkable criteria, and guide improvements within a fixed attempt limit. After a version passes, prepare separate evaluation cases to check performance beyond the examples used for revision.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Defining checkable criteria for agent outputs
• Improving an agent within a fixed attempt limit
• Testing a passing version on unseen cases

Evaluate Google Cloud workloads against built-in best practices or custom organizational rules using Workload Manager. Use public client libraries or the REST API to run checks, review violations and scanned resources, and export results to BigQuery.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Evaluate workloads against Google Cloud best practices
• Inspect rules for SAP and SQL Server workloads
• Run evaluations using custom organizational rules

Improve code, prompts, skills, or agent definitions through repeated changes and evaluation. Measure numerical targets such as latency or size, or use an LLM to judge pass-or-fail criteria, keeping changes that improve the result.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Improving page load performance
• Reducing build output size
• Refining prompts against evaluation criteria

Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fine-tuning an embedding model for domain-specific search
• Training a model to rerank search results
• Training SPLADE or ColBERT retrieval models

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fine-tuning an embedding model for domain-specific search
• Training a model to rerank search results
• Training SPLADE or ColBERT retrieval models

Evaluate code against a product requirements document, scoring implementation and tests separately for each acceptance criterion. Produce an evidence-backed final grade, identify gaps, and compare implementations using a fixed checklist and scoring method.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Checking whether a feature fulfills its requirements
• Comparing implementations of the same specification
• Auditing unit and end-to-end test coverage

Audit AGENTS.md, rules, skills, and their references for broken paths or commands. Optional independent agent reviews assess redundant instructions and practical usefulness, producing reports that identify what to retain, review, or consider trimming.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Checking paths and commands cited in agent instructions
• Finding redundant rules across skills and instructions
• Assessing which instruction files change agent behavior