“eval”的搜索结果

Build repeatable DeepEval tests for AI agents, chatbots, and applications that retrieve documents to answer questions. Reuse or generate evaluation datasets, inspect failures, and improve prompts, tools, or retrieval through repeated evaluation runs.

中文简介正在准备中。当前显示英语简介。

• Add repeatable quality tests for an AI agent
• Evaluate chatbot behavior across multiple conversation turns
• Generate evaluation datasets from documentation

Add DeepEval tracing to Python AI applications so execution steps appear in Confident AI. Choose a supported integration or manual @observe instrumentation, then capture inputs, outputs, step types, tags, and metadata.

中文简介正在准备中。当前显示英语简介。

• Trace execution steps in a Python AI application
• Select tracing integrations for an existing framework
• Add manual tracing with @observe

Set up a realtime evaluation folder with a suitable shared harness, the framework that runs evaluations. Prepare prompts, tool definitions, realistic starter data, and a README, then validate the setup with evaluation and test runs.

中文简介正在准备中。当前显示英语简介。

• Start a single-turn text-to-speech evaluation
• Set up evaluations using saved audio
• Build multi-turn simulations with mock tools

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Evaluate code generation models with HumanEval, MBPP, and multilingual benchmarks. The skill guides test execution and model comparisons using pass@k, an estimate of whether at least one of k generated solutions passes the tests.

中文简介正在准备中。当前显示英语简介。

• Compare code generation models on shared benchmarks
• Measure code generation performance across languages
• Evaluate instruction-tuned coding models

Evaluate Hugging Face Hub models using inspect-ai or lighteval, with local inference or a provider-backed option. Choose an inference engine, start with a small test, and switch engines when model compatibility requires it.

中文简介正在准备中。当前显示英语简介。

• Run benchmark evaluations on a local GPU
• Choose an evaluation framework and inference engine
• Check a small sample before a full evaluation

Add team-specific evaluation criteria to plugin-eval by designing a metric pack, a set of custom checks and measurements. Create its manifest and JSON-producing script, then run it through the evaluator using a consistent output structure.

中文简介正在准备中。当前显示英语简介。

• Add team-specific evaluation criteria
• Define custom checks and numerical metrics
• Build a metric-pack manifest and script

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

中文简介正在准备中。当前显示原文说明。

Evaluate AI agents by preparing test cases, scoring responses and tool use, investigating failures, and comparing results after changes. Supports built-in checks and custom scoring rules, including evaluations performed by a language model.

中文简介正在准备中。当前显示英语简介。

• Create test cases for an AI agent
• Check response quality and tool use
• Investigate failed evaluation cases

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

中文简介正在准备中。当前显示英语简介。

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Evaluate language models with over 100 benchmark tests covering knowledge, math, coding, safety, and image understanding. Configure runs on Docker, Slurm computing clusters, or cloud infrastructure, compare models, and export results.

中文简介正在准备中。当前显示英语简介。

• Compare models using the same benchmark tests
• Run large evaluations on a Slurm cluster
• Evaluate safety and image understanding

Evaluate language models on knowledge, math, coding, safety, and image understanding benchmarks. The skill guides reproducible runs in containers using local Docker, Slurm clusters, or cloud execution, with shared configurations for model comparison and result export.

中文简介正在准备中。当前显示英语简介。

• Comparing models on the same benchmark tasks
• Running a small evaluation sample before a full benchmark
• Running large evaluations on a Slurm cluster

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

中文简介正在准备中。当前显示英语简介。

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Audit a completed AI development phase to check whether its planned evaluations were implemented. Produce EVAL-REVIEW.md with a score, verdict, identified gaps, and a plan to address missing coverage.

中文简介正在准备中。当前显示英语简介。

• Review evaluation coverage after an AI phase is complete
• Check implemented evaluations against AI-SPEC.md
• Document evaluation gaps and remediation steps

Create Vally evaluation specifications for skills, custom agents, and workflow packages. Design distinct test requests, prepare fixture files, define automated checks and outcome rubrics, and size comparisons to provide meaningful evidence without favoring the skill’s wording.

中文简介正在准备中。当前显示英语简介。

• Create an eval.yaml for a skill or custom agent
• Define outcome-based checks and judging rubrics
• Prepare fixture files that reproduce specific failures

Provides context about the Roo Code evals system structure in this monorepo. Use when tasks mention "evals", "evaluation", "eval runs", "eval exercises", or working with the evals infrastructure. Helps distinguish between the evals execution system (packages/evals, apps/web-evals) and the public website evals display page (apps/web-roo-code/src/app/evals).

中文简介正在准备中。当前显示原文说明。

Find JavaScript that executes strings as code, including eval(), new Function(), and string-based timers. Review how it runs in the browser and suggest alternatives for parsing data, choosing operations, and scheduling functions to reduce code injection risks.

中文简介正在准备中。当前显示英语简介。

• Finding eval() and other string execution
• Replacing eval() used to parse JSON
• Changing string-based timers to function references

Create and run evaluations in Arize using an LLM as the judge. Define scoring criteria, connect them to actual data fields, evaluate recorded operations or experiment results, and set up continuous scoring for new data.

中文简介正在准备中。当前显示英语简介。

• Check answers for unsupported claims
• Score experiment results for correctness
• Continuously evaluate new production records

Configure OpenTelemetry to send AI application traces, records of execution steps, to Confident AI. Reuse existing exporters, choose the regional endpoint, and add AI-specific attributes without requiring the deepeval Python package.

中文简介正在准备中。当前显示英语简介。

• Send existing AI application traces to Confident AI
• Configure tracing for a non-Python AI application
• Choose the US or EU endpoint for an API key

Improve an evaluated Codex skill by turning plugin-eval findings into prioritized changes. Refine activation descriptions, shorten SKILL.md, move lengthy details into supporting files, fix structural issues, and compare evaluation results before and after the rewrite.

中文简介正在准备中。当前显示英语简介。

• Prioritizing fixes after a skill evaluation
• Shortening SKILL.md and reducing token use
• Clarifying when a skill should activate

使用统一任务评估大语言模型的知识、推理和代码生成能力，生成多模型对比表及训练阶段的成绩趋势，并保存评估结果和每道题的回答。

• 使用MMLU和GSM8K评估模型
• 生成多个模型的成绩对比表
• 跟踪训练期间的模型成绩

按目的选择Paperclip的Runner评估或完整产品端到端评估，结合证据、执行记录、费用及失败分类进行验证、结果解读和报告。

• 选择Runner或产品端到端评估
• 无需认证信息查看评估列表
• 依据证据与执行记录分析失败

INVOKE THIS SKILL for LLM-as-judge evaluation workflows on Arize: creating/updating evaluators, running evaluations on spans or experiments, tasks, trigger-run, column mapping, and continuous monitoring. Use when the user says: create an evaluator, LLM judge, hallucination/faithfulness/correctness/relevance, run eval, score my spans or experiment, ax tasks, trigger-run, trigger eval, column mapping, continuous monitoring, query filter for evals, evaluator version, or improve an evaluator prompt.

中文简介正在准备中。当前显示原文说明。

Plan AI application work across six areas: prompt evaluation, context allocation, retrieval-based answer generation, agent security, evaluation pipelines, and product strategy. This overview explains the approaches; the individual workflow packages, test cases, and CI templates are not included.

中文简介正在准备中。当前显示英语简介。

• Planning prompt evaluation before deployment
• Allocating context capacity and designing retrieval-based answers
• Planning an authorized AI agent security assessment