4514 个 · 按相关度排序
Design and implement RAG systems, which retrieve documents to inform AI answers. Get document preparation strategies, vector database comparisons, combined keyword and meaning-based search, result reranking, and an evaluation plan for retrieval quality.
中文简介正在准备中。当前显示英语简介。
Evaluate a local Codex skill for metadata, structure, reference links, size, and helper code quality. Get prioritized fixes and explanations, with benchmark setup for deeper analysis and measurement recommendations when observed usage is available.
中文简介正在准备中。当前显示英语简介。
Evaluate local Codex skills and plugins, understand their scores, and identify fixes to prioritize. Get explanations of token estimates, set up benchmark scenarios, and prepare workflows for measuring actual token usage.
中文简介正在准备中。当前显示英语简介。
Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine.
中文简介正在准备中。当前显示原文说明。
Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to evaluate its quality — separating retrieval issues from generation issues. Also use when the user mentions RAG evaluation, faithfulness checking, hallucination detection in RAG, retrieval quality, chunking optimization, or "is my RAG pipeline working." Outputs a diagnostic matrix that pinpoints whether problems are in retrieval or generation.
中文简介正在准备中。当前显示原文说明。
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.
中文简介正在准备中。当前显示原文说明。
Advanced Evaluation workflow skill. Use this skill when the user needs This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
中文简介正在准备中。当前显示原文说明。
Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns. Use when asked to "build an eval harness", "set up eval-as-code", or "version our eval datasets".
中文简介正在准备中。当前显示原文说明。
Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.
中文简介正在准备中。当前显示原文说明。
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
中文简介正在准备中。当前显示原文说明。
帮助设计让人工智能智能体持续工作的循环，结合质量检查、评估与失败后的停止和重试规则，处理进度停滞、反复失败及成本增长问题。
Evaluate BMad skills using your agent's configured runner. Compare results with the model alone or another version, grade outputs against criteria, and test whether descriptions trigger correctly, preserving transcripts and evidence for each run.
中文简介正在准备中。当前显示英语简介。
Evaluate how Chatbox answers questions about large attachments using synthetic and real documents. Check whether the model retrieves file content when needed, skips retrieval for unrelated requests, and investigate failures through the actual application flow.
中文简介正在准备中。当前显示英语简介。
Create evaluation drafts for existing skills using actual conversations and user corrections. Produce criteria and repeatable test cases, distinguish evidence from specification assumptions, and prepare an anonymized draft for human review.
中文简介正在准备中。当前显示英语简介。
Review completed AI work by scoring task difficulty and execution quality separately. A fixed scoring matrix, arguments for higher and lower ratings, and saved score history help examine weaknesses and detect repetitive scoring patterns across sessions.
中文简介正在准备中。当前显示英语简介。
Evaluate a machine learning GPU cluster during a cloud trial or acceptance test. Measure GPU computing performance, communication within and between machines, and local and shared storage, then compile the results into a dated Markdown report.
中文简介正在准备中。当前显示英语简介。
Evaluate a local Codex plugin to understand its scores and identify what to fix first. Review stronger and weaker skills, prepare benchmark runs, compare saved results across versions, and produce a readable report.
中文简介正在准备中。当前显示英语简介。
Diagnose why an existing skill fails evaluations or shows no clear improvement. Separate content problems from test data, execution failures, activation issues, and insufficient evaluation tasks, then apply targeted fixes and validate them.
中文简介正在准备中。当前显示英语简介。
Regenerate sounds in a draft PeonPing pack based on feedback. This internal workflow revises sound prompts or speech text, preserves the pack's style and voice, renders replacement audio, and records each successful change.
中文简介正在准备中。当前显示英语简介。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
中文简介正在准备中。当前显示原文说明。
Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization. Triggers on: "evaluate my agent", "add evaluator", "measure quality", "quality gate", "run evals", "agent too slow", "why is it slow", "reduce latency", "set up observability", "CloudWatch dashboard", "how much does my agent cost", "cost optimization", "logs not showing up", "logs missing", "spans not found", "eval failing", "eval error", "dev traces", "local traces", "agentcore dev traces", "traces to CloudWatch". Not for debugging errors or crashes — use agents-debug. Slow but correct routes here; broken routes to debug.
中文简介正在准备中。当前显示原文说明。
Design an evaluation plan for an LLM or AI feature before shipping it. Use when asked how to evaluate a prompt/model/agent, set up an eval harness, define quality metrics for an AI feature, or build a regression gate. Produces an eval plan — task definition, datasets, metrics & rubrics, baselines, automated + human evals, a pass bar, and a regression gate.
中文简介正在准备中。当前显示原文说明。
**WORKFLOW SKILL** - Evaluate AI agent skills using structured benchmarks with YAML specs, fixture isolation, and pluggable validators. USE FOR: run waza, waza help, run eval, run benchmark, evaluate skill, test agent, generate eval suite, init eval, compare results, score agent, agent evaluation, skill testing, cross-model comparison. DO NOT USE FOR: improving skill frontmatter (use waza dev), creating new skills from scratch (use skill-creator), token counting or budget checks (use waza tokens). INVOKES: Copilot SDK executor, mock engine, code/regex validators. FOR SINGLE OPERATIONS: use waza run directly for a single benchmark.
中文简介正在准备中。当前显示原文说明。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
中文简介正在准备中。当前显示原文说明。