“evals”的搜索结果

Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.

中文简介正在准备中。当前显示原文说明。

Claude API (platform.claude.com) のプロンプトエンジニアリング・評価リファレンス。 prompt engineering, XML タグ, few-shot, system prompts, モデル別プロンプティング（Fable 5.1 / Fable 5 / Opus 5.5 / Opus 5 / Sonnet 5.5 / Sonnet 5 / Haiku 5.5）。 evals, grading（LLM-graded 等）, jailbreak・prompt leak 対策, hallucination 低減, classification, ticket routing, commerce agent 等ユースケース実装。

中文简介正在准备中。当前显示原文说明。

Design a self-improvement loop for a shipped product's AI agents — production traces feed evals, evals feed improvement proposals (prompts, playbooks, retrieval configs), and a human approval gate promotes changes with rollback. Technique-agnostic: chooses per project between ACE-style evolving playbooks, GEPA/MIPROv2 offline optimization, or simple eval-driven iteration via references/techniques.md. Load when the user asks to make my product's agents self-improving, learn from production traces, add a learning loop, evolve prompts or playbooks safely, promote agent improvements, or GEPA-style optimization. NOT harness-evolution (that improves the coding agent), NOT experimentation (product A/B tests), NOT agent-run-retro (dev-phase manual retros — this skill is the production-scale continuation).

中文简介正在准备中。当前显示原文说明。

在AI开发前定义成功条件，使用代码、规则、模型或人工评估新功能并检查既有功能是否正常，再记录多次尝试的可靠性指标与报告，用于比较提示词、代理或模型版本的变化。

• 在AI实现前确定成功条件
• 检查提示词或代理变更后的回归问题
• 通过多次尝试衡量可靠性

将反复交给AI的任务整理为可复用技能，也可测试和改进已有技能。通过比较输出、收集用户意见和查看性能指标，修改技能指令及调用条件。

• 将重复任务整理为可复用技能
• 根据用户意见改进已有技能
• 比较使用与不使用技能的结果

查阅官方资料，提供附有来源的Codex设置、价格及OpenAI API与产品使用说明，也可用于模型选型、迁移规划、提示词调整和问题排查。

• 确认Codex设置并排查问题
• 比较模型的费用、速度和质量
• 规划模型迁移和提示词调整

为Paperclip Runner添加或扩展行为评估，定义判定条件、执行目标和报告验证数据，利用正常及异常证据验证结果，并保留执行记录与费用。

• 添加Runner行为评估
• 用有效与无效证据检验判定
• 更新执行列表和持续评估

Turn a repeatable workflow into a skill, or improve an existing one through test prompts and feedback. Compare results with and without the skill, review performance measurements, and refine its description so it triggers appropriately.

中文简介正在准备中。当前显示英语简介。

• Turn a recurring workflow into a reusable skill
• Improve an existing skill using feedback
• Compare outputs with and without a skill

Create and improve agent skills by defining tasks, writing instructions, and testing realistic requests. Compare outputs, success rates, time, and token use, then refine the skill and its description so it triggers appropriately.

中文简介正在准备中。当前显示英语简介。

• Turn a recurring workflow into a reusable skill
• Improve an existing SKILL.md using test feedback
• Compare results with and without a skill

Get guidance on Codex, OpenAI APIs, and ChatGPT Work using official documentation with citations. Use it to investigate settings and errors, compare models, plan migrations, or work on prompts and API integrations.

中文简介正在准备中。当前显示英语简介。

• Check Codex settings and troubleshoot errors
• Compare models for a particular task
• Plan a model migration and prompt changes

Build and maintain BMad skills, agents, and modules through a guided conversation. Turn prompts or session logs into reusable skills, review existing tools, and prepare modules for distribution, with approval before files are written.

中文简介正在准备中。当前显示英语简介。

• Create a skill from a prompt or session logs
• Build an agent with a defined persona
• Fix or review an existing skill

Evaluate BMad skills using your agent's configured runner. Compare results with the model alone or another version, grade outputs against criteria, and test whether descriptions trigger correctly, preserving transcripts and evidence for each run.

中文简介正在准备中。当前显示英语简介。

• Compare a skill with the model alone
• Compare an edited skill with a previous version
• Grade outputs against defined criteria

Create reusable agent skills or improve existing ones through test prompts and user feedback. Compare output quality, execution time, and token usage against a baseline, then refine instructions and descriptions so the skill is invoked in appropriate situations.

中文简介正在准备中。当前显示英语简介。

• Turn a recurring workflow into a reusable skill
• Improve an existing skill using test results
• Compare quality, runtime, and token usage across versions

Build repeatable evaluations for AI agent responses, tool choices, and multi-step tasks. Use reference datasets and scoring criteria to compare changes, detect quality regressions, and set automated checks that must pass before deployment.

中文简介正在准备中。当前显示英语简介。

• Compare agent quality before and after prompt changes
• Test tool selection and multi-step task execution
• Evaluate resistance to prompt injection

Design criteria for using an LLM to judge AI responses through scoring or comparisons between two answers. Includes checks for order and length bias, plus validation against human judgments to assess evaluation reliability.

中文简介正在准备中。当前显示英语简介。

• Build an automated evaluation pipeline for AI responses
• Compare responses after changing a prompt or model
• Define shared scoring criteria for reviewers

Turn recurring tasks into reusable agent skills, or improve skills you already have. Draft instructions, compare test outputs with and without a skill, review quality and resource use, and refine descriptions so skills trigger for appropriate requests.

中文简介正在准备中。当前显示英语简介。

• Turn a recurring workflow into a reusable skill
• Improve an existing skill using test results
• Compare output quality, runtime, and token use

Build and run evaluations for AI applications using Phoenix. Start with observed failures, create checks in code or use a language model as a judge, run experiments, and validate automated judgments against human labels.

中文简介正在准备中。当前显示英语简介。

• Turning observed failures into evaluation criteria
• Building code-based or language-model evaluators
• Evaluating retrieval and answer faithfulness in RAG

Investigate LLM application failures with Phoenix CLI by reviewing execution records, recording observations, and grouping mistakes into counted categories. Inspect datasets and experiments to identify which failure types deserve evaluation tests first.

中文简介正在准备中。当前显示英语简介。

• Investigating failures in LLM applications
• Grouping observed mistakes into failure categories
• Choosing which evaluation tests to build first

Build an end-to-end evaluation pipeline for Python AI applications using pixie-qa and real language-model calls. Define criteria, prepare realistic test data, score application runs, and analyze the evidence to produce a prioritized improvement plan.

中文简介正在准备中。当前显示英语简介。

• Setting up quality checks for Python AI applications
• Creating realistic scenarios for known failure modes
• Scoring full application runs with real language-model calls

Build and operate LLM applications with guidance on RAG, which uses retrieved documents to inform answers. Covers document indexing, vector database selection, prompt improvement, response evaluation, caching, and estimates of model usage costs.

中文简介正在准备中。当前显示英语简介。

• Building an assistant that answers from documents
• Choosing a vector database for document search
• Comparing answer quality after prompt changes

Validates skill files and registration consistency, then checks project tests and system health. Compares failures with recent changes to distinguish regressions, outdated tests, intermittent failures, and infrastructure issues, producing a report that tracks results over time.

中文简介正在准备中。当前显示英语简介。

• Validating definitions and registration for new skills
• Running daily tests and system health checks
• Classifying failures after code changes

Build, edit, or debug backend AI agents with eve, a framework that represents agents as directories of files. Work on instructions, tools, connections, schedules, and related components using documentation bundled with the installed version.

中文简介正在准备中。当前显示英语简介。

• Create a backend AI agent with eve
• Update agent instructions, tools, and connections
• Debug schedules and subagent configurations

Strengthen existing tests with fast-check by checking properties across generated inputs. Find bugs that example tests miss and reduce failing inputs to small, reproducible cases. Property testing is available; the other listed workflows remain planned.

中文简介正在准备中。当前显示英语简介。

• Checking parser behavior across generated inputs
• Testing serialization and deserialization round trips
• Finding pure functions covered only by example tests

Develop varied ideas for names, writing angles, and problem solving by exploring five candidates internally and presenting the strongest. It also expands small example collections into diverse datasets for evaluation, training, or testing.

中文简介正在准备中。当前显示英语简介。

• Finding a distinctive product name
• Developing a fresh angle for a blog post
• Exploring solutions to a technical problem