“evaluation”的搜索结果

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when "langfuse, llm observability, llm tracing, prompt management, llm evaluation, monitor llm, debug llm, langfuse, observability, tracing, llm-monitoring, evaluation, prompt-management, debugging, analytics" mentioned.

中文简介正在准备中。当前显示原文说明。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarksUse when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" mentioned.

中文简介正在准备中。当前显示原文说明。

Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

中文简介正在准备中。当前显示原文说明。

Design a source evaluation protocol using lateral reading and credibility checks for digital information. Use when students need to evaluate websites, online sources, or social media claims.

中文简介正在准备中。当前显示原文说明。

LLM-as-judge evaluation framework with 5-dimension rubric (accuracy, groundedness, coherence, completeness, helpfulness) for scoring AI-generated content quality with weighted composite scores and evidence citations

中文简介正在准备中。当前显示原文说明。

Generate msmodelslim evaluation YAML configuration (service_oriented + aisbench + vllm-ascend). Use when user asks for evaluation config generation.

中文简介正在准备中。当前显示原文说明。

Create evaluation frameworks for AI outputs with rubrics and scoring. TRIGGERS - Use when user needs help with ai-evaluation-framework related tasks.

中文简介正在准备中。当前显示原文说明。

Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.

中文简介正在准备中。当前显示原文说明。

Use when designing evaluation suites for agentic systems, choosing agent benchmarks, measuring multi-agent performance, or critiquing single-score evaluation of agents. Based on arXiv:2605.26112v1. Trigger phrases: "evaluate my agent", "benchmark agent", "measure agent reliability", "pass^k evaluation", "agent metrics beyond success rate", "trajectory quality", "memory hygiene metric", "longitudinal agent eval", "why my agent scores well on benchmarks but fails in production".

中文简介正在准备中。当前显示原文说明。

Turns a business need and its constraints into a DRAFT RFP requirements pack: traced functional and non-functional requirements with priorities, mandatory gates, evaluation criteria with weights left to the business owner, and fact-seeking supplier questions. Solution-neutral; never names a supplier or product. Use when the user asks to "write the RFP requirements", "turn this business case into tender requirements", "draft the supplier questionnaire", "structure our RFQ from this brief" or "set up evaluation criteria before we go to market". Do not use for responses already received, use rfp-comparison-pack to compare them or supplier-evaluation-matrix to score them instead. Drafts for human review; never approves, authorises or signs off.

中文简介正在准备中。当前显示原文说明。

Builds a DRAFT weighted supplier evaluation matrix from supplier responses and the panel's agreed criteria, weights and scale: a provisional score per cell with the quoted evidence and anchor behind it, a register of every missing or partial answer, and calibration and sensitivity flags. Provisional for panel moderation; never recommends, awards or disqualifies. Use when the user asks to score, weight or rank supplier, bid or tender responses against agreed criteria, for example "score these bids", "build the evaluation matrix", "weight the tender responses" or "rank the suppliers against our criteria". Do not use for a side-by-side comparison without scores, use rfp-comparison-pack instead; for writing the criteria, use rfp-requirements-pack instead. Drafts for human review; never approves, authorises or signs off.

中文简介正在准备中。当前显示原文说明。

Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

中文简介正在准备中。当前显示原文说明。

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

中文简介正在准备中。当前显示原文说明。

評価統合エージェント - MMI評価とDDD評価の結果を統合し、優先度付き改善計画を策定。/integrate-evaluations [対象パス] で呼び出し。

中文简介正在准备中。当前显示原文说明。

DDD評価エージェント - 戦略的設計・戦術的設計の両面からDDD原則への適合度を評価し改善点を特定。/ddd-evaluation [対象パス] で呼び出し。

中文简介正在准备中。当前显示原文说明。

Generate HTML reports and structured metrics from model evaluation results. Creates publication-ready reports with embedded outputs (images, audio, charts) and metrics.json for cross-model comparison. Use when generating reports, writing metrics, creating evaluation summaries, or formatting benchmark results. Triggers on "generate report", "write metrics", "create report", "evaluation summary", "benchmark results", "format results".

中文简介正在准备中。当前显示原文说明。

Focused Signals scout for PostHog projects using feature flags. Watches the flag roster and the `$feature_flag_called` evaluation stream for contradictions between a flag's configured state and its real traffic: evaluation cliffs on healthy flags, ghost flags (code calling keys that no longer exist), response-distribution shifts with no corresponding flag edit, and flag debt (stale, fully-rolled-out, or dead flags still burning evaluations). Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills.

中文简介正在准备中。当前显示原文说明。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

中文简介正在准备中。当前显示原文说明。

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

中文简介正在准备中。当前显示原文说明。

Generate the success criteria for a task or question, then review work against them. Given a task, goal, or open-ended question, decompose it into scenarios, evaluation perspectives, and fine-grained weighted YES/NO criteria using the Recursive Expansion Tree (RET) method; if work is supplied, score it criterion-by-criterion and surface what is missing or could be better. Use when asked to self-review or check your own work, judge whether a task is done well or completely, build a definition-of-done or completeness checklist, create an evaluation rubric or grading criteria, score or grade answers to a question, set up an LLM-as-judge rubric, or when the user mentions self-review, completeness check, success criteria, evaluation criteria, scoring rubric, Qworld, or the RET algorithm.

中文简介正在准备中。当前显示原文说明。

Create evaluation frameworks for AI outputs with rubrics and scoring. TRIGGERS - Use when user needs help with ai-evaluation-framework related tasks.

中文简介正在准备中。当前显示原文说明。

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

中文简介正在准备中。当前显示原文说明。

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

中文简介正在准备中。当前显示原文说明。

Plan, evaluate, and adopt agent infrastructure for a named workload. Use for agent runtime/framework selection, agent evaluation or observability plans, memory and tool boundaries, infrastructure cost controls, and organizational adoption. Covers workload-first comparison, risk and recovery planning, held-out evaluation, cost accounting, and staged rollout. Not a vendor ranking or a production certification. NOT for implementing a specific agent behavior (agentic-patterns), building a specific agent (ai-engineer), prompt optimization (prompt-engineer), or conventional ETL workflow design (jury_rig-architect).

中文简介正在准备中。当前显示原文说明。