3.6K skills · sorted by relevance
Build an end-to-end evaluation pipeline for Python AI applications using pixie-qa and real language-model calls. Define criteria, prepare realistic test data, score application runs, and analyze the evidence to produce a prioritized improvement plan.
Run repeated change-and-evaluation cycles for one predefined improvement mission within an explicit runtime limit. Save each evaluation as JSON and each decision in readable Markdown logs, with support for periodic reruns through Claude Code native cron.
Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.
Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.
Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.
Direct finance questions to the appropriate specialist skill: financial statement analysis or SaaS operating metrics. It helps choose the right workflow for valuation, budgets, recurring revenue, churn, and customer acquisition costs.
Evaluate Google Cloud workloads against built-in best practices or custom organizational rules using Workload Manager. Use public client libraries or the REST API to run checks, review violations and scanned resources, and export results to BigQuery.
Evaluate AI models and agents on Google Cloud using test datasets and scoring criteria. Generate simulated user scenarios, analyze failures, and compare results across changes to guide improvements to prompts and code.
Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.
Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.
Evaluate Hugging Face Hub models using inspect-ai or lighteval, with local inference or a provider-backed option. Choose an inference engine, start with a small test, and switch engines when model compatibility requires it.
Audit a completed AI development phase to check whether its planned evaluations were implemented. Produce EVAL-REVIEW.md with a score, verdict, identified gaps, and a plan to address missing coverage.
Evaluate a local Codex plugin to understand its scores and identify what to fix first. Review stronger and weaker skills, prepare benchmark runs, compare saved results across versions, and produce a readable report.
Evaluate a local Codex skill for metadata, structure, reference links, size, and helper code quality. Get prioritized fixes and explanations, with benchmark setup for deeper analysis and measurement recommendations when observed usage is available.
Evaluate staged Git changes before committing using an LLM, a language model acting as a reviewer. Receive correctness, completeness, and safety scores, plus issues by severity, to decide whether to fix, commit, or stop.
Diagnose why an existing skill fails evaluations or shows no clear improvement. Separate content problems from test data, execution failures, activation issues, and insufficient evaluation tasks, then apply targeted fixes and validate them.
Manage crypto asset cost basis in TRES Finance, from choosing how acquisition costs are assigned to reviewing realized gains and losses. Recalculate results, investigate missing prices, manage revaluations, and export cost basis reports.
Develop a broad research interest into evaluated project options. Specialized agents generate ideas, assess feasibility and competition, and draft abstracts and conclusions, helping researchers decide what to pursue, postpone, or drop and which assumption to test first.
Find the right code for Roo Code evaluations, which test AI coding tasks. This skill explains the execution system, management interface, and public results page, helping agents identify where changes and debugging work belong.
Use when packaging ACM CCS artifacts for the artifact-evaluation committee and the ACM badges — Artifacts Available, Artifacts Evaluated Functional, Artifacts Evaluated Reusable, and Results Reproduced — covering what security evaluators inspect, how to make attacks and defenses turnkey, and how to justify withheld artifacts.
The English summary is being prepared. Showing the original description.
RFP creation, distribution, evaluation, and shortlisting for in-house legal teams selecting outside counsel. Draft a new legal services RFP from scratch, evaluate firm responses against weighted criteria, produce a shortlist recommendation with selection rationale for GC sign-off, or design the end-to-end RFP process from scratch. Trigger on: 'draft an RFP', 'run an RFP process', 'evaluate firm responses', 'score the RFP submissions', 'which firms should we shortlist', 'selection recommendation', 'how do we run a panel RFP', 'RFP for legal services', 'pitch process', 'firm selection process', 'compare the proposals', 'we're reviewing our panel', 'panel refresh RFP', 'we need to go to market', 'request for proposal', 'evaluate the pitches', 'which firm won the RFP', 'write a legal RFP', 'RFP evaluation criteria', 'design the RFP process', 'how do we weight the criteria'.
The English summary is being prepared. Showing the original description.
Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.
The English summary is being prepared. Showing the original description.
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
The English summary is being prepared. Showing the original description.
Guide for diagnosing and improving MSBuild project evaluation performance. USE FOR: builds slow before any compilation starts, high evaluation time in binlog analysis, expensive glob patterns walking large directories (node_modules, .git, bin/obj), deep import chains (>20 levels), preprocessed output >10K lines indicating heavy evaluation, property functions with file I/O ($([System.IO.File]::ReadAllText(...))), multiple evaluations per project. Covers the 5 MSBuild evaluation phases, glob optimization via DefaultItemExcludes, import chain analysis with /pp preprocessing. DO NOT USE FOR: compilation-time slowness (use build-perf-diagnostics), incremental build issues (use incremental-build), non-MSBuild build systems.
The English summary is being prepared. Showing the original description.