1.3K skills · sorted by relevance
Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.
Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.
Supports preparation for EU medical device regulation reviews by organizing device classification, technical documentation, clinical evidence, and post-market monitoring. Includes a tool to identify documentation and compliance gaps and prioritize the next steps.
Evaluate Google Cloud workloads against built-in best practices or custom organizational rules using Workload Manager. Use public client libraries or the REST API to run checks, review violations and scanned resources, and export results to BigQuery.
Evaluate AI models and agents on Google Cloud using test datasets and scoring criteria. Generate simulated user scenarios, analyze failures, and compare results across changes to guide improvements to prompts and code.
Guides machine learning work from data preparation and model training to evaluation and production monitoring. Covers PyTorch, scikit-learn, experiment tracking, and reproducible workflows so teams can compare models and maintain them after deployment.
Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.
Guides iterative improvement of AI agent prompts, skills, and memory using A-Evolve. It explains how to evaluate agents against benchmark tasks, apply changes, validate results, and track or roll back revisions with Git.
Design and improve prompts for language models, then test their accuracy and consistency. Create reusable templates, JSON output schemas that define response structure, evaluation criteria, and test cases while considering token use and model differences.
Build evaluation suites for AI features with the Axiom AI SDK. Generate test files, scoring functions, test cases, and configurable parameters from descriptions and existing code to compare changes and monitor output quality in production.
Execute an existing Ouroboros Seed, a specification containing a goal and acceptance criteria, as a background job. The workflow checks each criterion, normally queues formal evaluation, and follows a bounded improvement loop if evaluation rejects the result.
Evaluate AI agents by preparing test cases, scoring responses and tool use, investigating failures, and comparing results after changes. Supports built-in checks and custom scoring rules, including evaluations performed by a language model.
Create Vally evaluation specifications for skills, custom agents, and workflow packages. Design distinct test requests, prepare fixture files, define automated checks and outcome rubrics, and size comparisons to provide meaningful evidence without favoring the skill’s wording.
Manage cost basis calculation in TRES Finance: check status, view/change strategy (FIFO, LIFO, AVG, MAX_GAINS, MAX_LOSSES, FIFO_IMPAIRMENT), trigger recalculation, review financial issues, manage reevaluations/impairments, set bulk fiat prices, configure spec-ID lot disposal rules, view per-asset cost basis results, and export cost basis reports. Trigger this skill whenever the user asks about cost basis, gains/losses, realized gains, unrealized gains, COGS, inventory lots, tax lots, cost basis strategy, reevaluation, impairment, fiat value override, spec-ID rules, financial issues (negative balance, missing fiat), or anything related to how their crypto assets are costed. Also trigger when the user says "calculate cost basis", "what is my cost basis strategy", "show me financial issues", "create a reevaluation", "run cost basis", "change strategy to LIFO", or similar.
The English summary is being prepared. Showing the original description.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
The English summary is being prepared. Showing the original description.
Design evaluation datasets and supporting documentation for AI systems and agents, covering the scoping interview, difficulty distribution, dataset contract, sample review, and metric and tooling selection. Use when building or reviewing an evaluation set for a conversational agent, assistant, or retrieval-grounded AI system.
The English summary is being prepared. Showing the original description.
Use when evaluating the model and analyzing results for an Agricultural Systems (AgSy) manuscript so it survives expert systems review — independent model evaluation (observed vs. simulated, fit statistics), sensitivity and uncertainty analysis, and trade-off / scenario analysis across the system. Guides evaluation norms; it does not fabricate results or run the model.
The English summary is being prepared. Showing the original description.
Evaluate, select, and contract with vendors and SaaS tools. Use this skill when comparing alternatives, running an RFP, scoring vendors against criteria, negotiating contracts, planning a switch, or assessing a vendor's risk. Triggers on vendor evaluation, RFP, vendor selection, build vs buy, SaaS evaluation, vendor scorecard, vendor comparison, contract negotiation, vendor switch, procurement. Also triggers when a renewal is coming up or when a tool isn't meeting expectations.
The English summary is being prepared. Showing the original description.
Design a source evaluation protocol using lateral reading and credibility checks for digital information. Use when students need to evaluate websites, online sources, or social media claims.
The English summary is being prepared. Showing the original description.
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.
The English summary is being prepared. Showing the original description.
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
The English summary is being prepared. Showing the original description.
Provides weighted scoring, rubrics, and decision-threshold patterns. Use when designing quality gates, evaluation systems, or decision frameworks.
The English summary is being prepared. Showing the original description.
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
The English summary is being prepared. Showing the original description.
Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.
The English summary is being prepared. Showing the original description.