4.5K skills · sorted by relevance
Analyze LLM eval results — score breakdowns by category, regression detection vs baseline, failure clustering. Use when asked to "analyze our eval results", "did the model regress", or "cluster the eval failures".
The English summary is being prepared. Showing the original description.
An AI analyst for buying B2B software. Investigates what vendors claim, checks the evidence, surfaces contradictions and unanswered questions, and produces a shareable Decision Brief with the questions to ask before you buy. Quick Eval by default; Deep Eval on request. Use when asked to evaluate, compare, vet, or research B2B software vendors.
The English summary is being prepared. Showing the original description.
Create and maintain AgentV YAML evaluation files for testing AI agent performance. Use this skill when creating new eval files, adding eval cases, or configuring custom evaluators (code validators or LLM judges) for agent testing workflows.
The English summary is being prepared. Showing the original description.
Use when testing, evaluating, or building regression suites for Agentforce agents: conversation testing in Agent Builder, topic (now subagent) coverage and utterance testing, Testing API and AiEvaluationDefinition metadata, Agentforce DX CLI test runs (sf agent generate test-spec, sf agent test create/run/resume/results/list), evaluation metrics (containment rate, escalation rate, CSAT, topic activation accuracy), and post-deploy analytics via Enhanced Event Logs. Triggers: 'how do I test my Agentforce agent', 'agent routes to wrong subagent', 'write utterance tests', 'regression test after topic change', 'measure agent quality', 'agent containment rate', 'run agent tests from the CLI'. NOT for agent creation, topic design, or action contract design — use agentforce/agentforce-agent-creation, agentforce/agent-topic-design, or agentforce/agent-actions respectively.
The English summary is being prepared. Showing the original description.
Avalia LLMs em mais de 60 benchmarks acadêmicos (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use quando benchmarking de qualidade de modelo, comparação de modelos, relatório de resultados acadêmicos ou rastreamento de progresso de treinamento. Padrão da indústria usado por EleutherAI, HuggingFace e laboratórios principais. Suporta HuggingFace, vLLM, APIs.
The English summary is being prepared. Showing the original description.
Use when designing evaluation suites for agentic systems, choosing agent benchmarks, measuring multi-agent performance, or critiquing single-score evaluation of agents. Based on arXiv:2605.26112v1. Trigger phrases: "evaluate my agent", "benchmark agent", "measure agent reliability", "pass^k evaluation", "agent metrics beyond success rate", "trajectory quality", "memory hygiene metric", "longitudinal agent eval", "why my agent scores well on benchmarks but fails in production".
The English summary is being prepared. Showing the original description.
NVIDIA RAG Blueprint evaluation guidance for measuring retrieval and answer quality with stable datasets, baselines, and reproducible scoring workflows.
The English summary is being prepared. Showing the original description.
Evaluate recommender system quality for the CSX4207 Vinyl Record Store. Use whenever you must compute or report recommender metrics (Precision@k, Recall@k, HitRate@k, MRR, MAP@k, NDCG@k, coverage, diversity, novelty, serendipity, personalization), design an evaluation protocol/split, run a baseline comparison, or write the evaluation section of a course deliverable. Covers formulas, JavaScript reference implementations, and the reporting checklist.
The English summary is being prepared. Showing the original description.
Expert en évaluation d'agents IA (benchmarking, human eval, latence, coût, précision, safety testing)
The English summary is being prepared. Showing the original description.
Guide for evaluation — inline evaluators, Inspect AI integration, and custom evaluators for measuring training progress. Use when the user asks about evaluation, metrics, benchmarks, or how to measure model quality during training.
The English summary is being prepared. Showing the original description.
Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
The English summary is being prepared. Showing the original description.
Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.
The English summary is being prepared. Showing the original description.
OpenAI API (developers.openai.com) の Evals / Fine-tuning リファレンス。 Evals API, Datasets, graders（string check / text similarity / model / python / multi）, agent evals, trace grading, external models, Promptfoo（移行先）, red teaming, SFT, DPO, RFT, vision fine-tuning, distillation, prompt optimizer, fine-tuning best practices。 Evals は 2026-11-30 停止予定、セルフサービス fine-tuning は段階的廃止中。
The English summary is being prepared. Showing the original description.
Set up eval-based QA for Python LLM applications: instrument the app, build golden datasets, write and run eval tests, and iterate on failures. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.
The English summary is being prepared. Showing the original description.
Reviews papers, research proposals, and literature reviews using a consistent scoring guide. Examines methods, evidence, citations, and writing, then separates serious problems from revision suggestions and identifies concrete next edits.
Develop applications through separate planning, building, and evaluation agents. The workflow tests the running app, scores its design and functionality, and feeds specific issues back into repeated development cycles with configurable quality thresholds and iteration limits.
Plan electricity and natural gas purchasing for commercial and industrial sites. Compare supplier offers, analyze tariffs and peak-demand charges, evaluate renewable energy contracts, and build budgets that account for price uncertainty and sustainability goals.
Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.
Design evaluation tasks that check whether agents actually complete work. Define requests, simulated user behavior, starting conditions, and verifiable success criteria, then plan difficulty levels, training-data leakage checks, and ongoing maintenance.
Build and evaluate single-agent reinforcement learning experiments, where an agent learns actions from rewards. Use Stable Baselines3 to choose algorithms, create environments, monitor training, and save models with the normalization data needed for consistent evaluation.
Build repeatable evaluations for AI agent responses, tool choices, and multi-step tasks. Use reference datasets and scoring criteria to compare changes, detect quality regressions, and set automated checks that must pass before deployment.
Design criteria for using an LLM to judge AI responses through scoring or comparisons between two answers. Includes checks for order and length bias, plus validation against human judgments to assess evaluation reliability.
Design agent memory that preserves facts, preferences, and conversation history across sessions. Plan storage, retrieval, and retention separately for each organization or user, with checks for unauthorized access, outdated information, conflicting records, and retrieval quality.
Turn graded execution traces and production logs into model training examples and preferred-versus-rejected response pairs. Select useful records, remove sensitive information, exclude evaluation examples, and preserve links to the original runs.