3,736 件 ・ 関連度順
概要と使いどころ
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
日本語の概要は準備中です。原文の説明を表示しています。
システムテスト・受入テスト（UAT）の仕様書をテンプレートから生成する。 「システムテスト仕様書を作って」「UATを準備して」「受入テストの仕様書が必要」などのリクエストで使用する。
Guide for evaluation — inline evaluators, Inspect AI integration, and custom evaluators for measuring training progress. Use when the user asks about evaluation, metrics, benchmarks, or how to measure model quality during training.
日本語の概要は準備中です。原文の説明を表示しています。
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
日本語の概要は準備中です。原文の説明を表示しています。
Runs the CARLA Leaderboard evaluator over a set of routes with your agent — routes/routes-subset/repetitions/track selection, checkpoint and live-results endpoints, resume after a crash, recording, and the debug levels — for leaderboard 1.0, 2.0 or 2.1. Preflights the whole stack (version pairing, maps, ports, agent class, sensor budget) before spending hours, and hands the world back in async mode. Use when the user asks to "run the leaderboard", "evaluate my agent", "run routes_training", "resume an evaluation", or "reproduce my submission locally".
日本語の概要は準備中です。原文の説明を表示しています。
Technical research methodology with YAGNI/KISS/DRY principles. Phases: scope definition, information gathering, analysis, synthesis, recommendation. Capabilities: technology evaluation, architecture analysis, best practices research, trade-off assessment, solution design. Actions: research, analyze, evaluate, compare, recommend technical solutions. Keywords: research, technology evaluation, best practices, architecture analysis, trade-offs, scalability, security, maintainability, YAGNI, KISS, DRY, technical analysis, solution design, competitive analysis, feasibility study. Use when: researching technologies, evaluating architectures, analyzing best practices, comparing solutions, assessing technical trade-offs, planning scalable/secure systems.
日本語の概要は準備中です。原文の説明を表示しています。
Planner→Builder→EvaluatorのLong-Runningエージェントハーネスで複雑なタスクを実装します。plannerがfeatures.jsonに機能を分解し、builderがTDDで1機能ずつ実装し、evaluatorがPASSするまでbuilderを繰り返します。「〇〇を作って」「実装して」「long-running」などのリクエストで起動します。
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
日本語の概要は準備中です。原文の説明を表示しています。
Validate process decomposition and architecture design quality before execution begins. Load when the setup-evaluator agent fires (automatic for agent-chain tasks), or when user says "evaluate this setup", "check the decomposition", "validate the architecture", "is this plan sound", "review the agent design". Catches structural errors, missing knowledge, unrealistic step ordering, and topology mismatches. Does NOT modify — only evaluates.
日本語の概要は準備中です。原文の説明を表示しています。
Orchestrator for the pre-decision business-idea lifecycle — generate ideas, model them, evaluate them, validate them with customers, and only then hand off to product-soul / brainstorming. Routes through `idea-generation`, `business-modeling`, `idea-evaluation`, and `customer-discovery`. Load when the user asks to explore business ideas, find a startup idea, evaluate a venture, validate an idea, says "what business should I start", "should I build this", "is this a good business", "I have a startup idea", "evaluate this venture", "model this business", "validate this idea", "Mom Test this", "Lean Canvas this", "Business Model Canvas", "Value Proposition Canvas", "go/no-go on this idea". Pre-decision suite — once one idea is committed, hands off to `product-soul`. Does NOT design features (use `brainstorming`) or audit built products (use `reality-check`).
日本語の概要は準備中です。原文の説明を表示しています。
Score an unbuilt business idea on desirability, viability, feasibility, distribution wedge, why-now, founder-market-fit, market size, alternatives, defensibility, capital intensity, and regulatory/ethical risk — and return a GO / ITERATE / KILL verdict with kill criteria and a next kill test. Load when the user asks to evaluate a business idea, score a startup idea, screen an idea, decide whether to pursue this venture, do an idea review, or says "is this a good business idea", "should I build this", "evaluate this startup", "screen this idea", "go/no-go on this idea", "kill or pursue". Sub-skill of `venture-exploration`. Calls `fermi` for sizing, `assumption-mapping` for hidden beliefs, optional `pre-mortem` / `adversarial-hat` for high-stakes ideas. Does NOT evaluate built products — for that use `reality-check`.
日本語の概要は準備中です。原文の説明を表示しています。
Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.
日本語の概要は準備中です。原文の説明を表示しています。
Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.
日本語の概要は準備中です。原文の説明を表示しています。
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
日本語の概要は準備中です。原文の説明を表示しています。
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
日本語の概要は準備中です。原文の説明を表示しています。
Detects typosquatting attacks in npm and PyPI package registries by analyzing package name similarity using Levenshtein distance and other string metrics, examining publish date heuristics to identify recently created packages mimicking established ones, and flagging download count anomalies where suspicious packages have disproportionately low usage compared to their legitimate targets. The analyst queries the PyPI JSON API and npm registry API to gather package metadata for automated comparison. Activates for requests involving package typosquatting detection, dependency confusion analysis, malicious package identification, or software supply chain threat hunting in package registries.
日本語の概要は準備中です。原文の説明を表示しています。
Decision tree analysis for complex decision-making across all domains. Use when user needs to evaluate multiple options with uncertain outcomes, assess risk/reward scenarios, or structure choices systematically. Applicable to business, investment, personal decisions, operations, career choices, product strategy, and any situation requiring structured evaluation. Triggers include decision tree, should I, what if, evaluate options, compare alternatives, risk analysis.
日本語の概要は準備中です。原文の説明を表示しています。
Rigorous benchmarking of multi-agent coordination systems: experiment design, statistical analysis, human evaluation protocols, and reproducible reporting. NOT FOR ML model evaluation (use llm-evaluation-harness), A/B testing for web products, survey design, or general data science.
日本語の概要は準備中です。原文の説明を表示しています。
Plan, evaluate, and adopt agent infrastructure for a named workload. Use for agent runtime/framework selection, agent evaluation or observability plans, memory and tool boundaries, infrastructure cost controls, and organizational adoption. Covers workload-first comparison, risk and recovery planning, held-out evaluation, cost accounting, and staged rollout. Not a vendor ranking or a production certification. NOT for implementing a specific agent behavior (agentic-patterns), building a specific agent (ai-engineer), prompt optimization (prompt-engineer), or conventional ETL workflow design (jury_rig-architect).
日本語の概要は準備中です。原文の説明を表示しています。
予測市場の価格が示す確率を出典と時刻付きで調べ、取引量や結果の判定ルール、他の情報源との比較から、製品や業務の判断材料として使えるか評価するスキル。
類似企業の売上・利益率・評価倍率を比較するExcelを作成します。計算式、中央値や四分位、データの出典をまとめ、投資や買収の検討に使う比較資料を整えます。
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
日本語の概要は準備中です。原文の説明を表示しています。