384 件 ・ 関連度順
概要と使いどころ
Use to consolidate multiple already-run judge verdicts into one report — consensus, conflicts, must-fix/should-fix with per-judge provenance. Consume-only, no opaque score, never auto-gates.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff may lack tests — missing assertions, uncovered branches, over-mocking, no regression test for a bug fix — dispatched by /review-changes, /do-and-judge, /judge, even without 'tests'.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff needs a readability review — naming, single-responsibility, DRY, dead code, mismatch with codebase conventions — dispatched by /review-changes, /do-and-judge, /judge.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff needs a requirement review — does it satisfy every stated acceptance criterion — dispatched by /review-changes, /do-and-judge, /judge. Never infers criteria from the diff.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff may introduce security risk — authZ, injection, secrets, unsafe deserialization, SSRF, XSS, mass assignment — dispatched by /review-changes, /do-and-judge, /judge.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluate LLM outputs using frontier models as judges. Use for pairwise model comparison, quality scoring with custom rubrics, and automated evaluation pipelines. Covers position bias mitigation, statistical significance, and generating preference data for DPO/RLHF.
日本語の概要は準備中です。原文の説明を表示しています。
Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
日本語の概要は準備中です。原文の説明を表示しています。
Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.
日本語の概要は準備中です。原文の説明を表示しています。
Guide the user through an independent iFixAi audit of their own agent, checking whether it does the job it is supposed to do given their business rules and org structure. Prefer pointing it at the user's REAL deployed agent over its HTTP endpoint (its actual tools, retrieval, and governance) with `--provider http --endpoint <url>`; only when no endpoint is reachable, fall back to replicating the model beneath as a bare stand-in (Anthropic, OpenAI, Gemini, Azure, Bedrock, etc.). Graded by the judge(s) of their choice (the same model, one independent judge, or a cross-vendor panel). You are the operator who walks them through it and explains the scorecard, running the SAME `ifixai run` engine as the guided CLI. Use when the user asks to run iFixAi or to audit an agent.
日本語の概要は準備中です。原文の説明を表示しています。
Monthly close runbook: how to walk {{close_sheet}}'s checklist, reconcile Stripe revenue against the recorded ledger, judge a missing supporting document, and spot an anomaly against prior months. Load this before touching a single checklist line so every item is judged to the same standard and only genuine open items reach a human.
日本語の概要は準備中です。原文の説明を表示しています。
Assertion-first AI eval framework aligned to Anthropic's 'Demystifying evals for AI agents' — typed deterministic asserts + a forced-structured LLM judge over an input→assert case schema, pass^k/pass@k, capability vs regression suites, subscription-billed. USE WHEN eval, evaluate, benchmark, regression test, assertion, assert, llm-rubric, judge, pass@k, pass^k, grade output, compare prompts/models, test agent. NOT FOR scientific-method framing (use Science), property/mutation testing of code (use Hardening), or live UI verification (use Interceptor).
日本語の概要は準備中です。原文の説明を表示しています。
Run one unattended IDEATION iteration of the autonomous value-creation loop — invent improvements a user of cc-wf-studio would notice, judge them against the value bar, and file the winners as locked `idea` issues. Never implements anything; the next-task skill builds from the queue this skill fills. Use when the user says "アイデア出して", "next idea", or wants proposals without implementation.
Run one unattended IDEATION iteration of the quality-assurance loop — find the highest-value untested behavior in the codebase, judge it against the QA value bar, and file ONE locked `qa` issue specifying the test to write. Never writes code or tests; the next-qa skill builds from the queue this skill fills. Use when the user says "QAアイデア", "next qa idea", or wants the QA backlog refilled without implementation.
Execute a task with sub-agent implementation and LLM-as-a-judge verification with automatic retry loop
日本語の概要は準備中です。原文の説明を表示しています。
Execute tasks through competitive multi-agent generation, meta-judge evaluation specification, multi-judge evaluation, and evidence-based synthesis
日本語の概要は準備中です。原文の説明を表示しています。
Evaluate solutions through multi-round debate between independent judges until consensus
日本語の概要は準備中です。原文の説明を表示しています。
Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vanity metrics, etc. Use when inheriting an eval system, when unsure whether evals are trustworthy, or as a starting point when no eval infrastructure exists. Do NOT use when the goal is to build a new evaluator from scratch (use error-discovery, write-judge-prompt, or validate-evaluator instead).
日本語の概要は準備中です。原文の説明を表示しています。
This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs pairwise", or "measure agent quality".
日本語の概要は準備中です。原文の説明を表示しています。
Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_sciences. Use when the user asks to review, evaluate, critique, or assess a research paper, check references, or verify a BibTeX file.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user has changed a prompt (system prompt, RAG template, agent instruction, etc.) and wants to know whether the candidate is better or worse than the baseline. Also use when the user mentions prompt A/B testing, prompt comparison, prompt optimization validation, "did my prompt change help," or prompt regression testing. Outputs per-dimension win rates with statistical significance using OpenJudge PairwiseAnalyzer.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user has nothing — no traces, no labels, no eval set — and needs to build a v0 evaluation from scratch. Also use when the user says "I need to start evaluating my app but don't know where to begin," "I want to set up eval for a new product," or has just identified failure modes and needs to turn them into principles. Outputs a v0 grader in 30 minutes using OpenJudge SimpleRubricsGenerator, plus a roadmap to reach calibrated evaluation.
日本語の概要は準備中です。原文の説明を表示しています。
Build RL reward signals using the OpenJudge framework. Covers choosing between pointwise and pairwise reward strategies based on RL algorithm, task type, and cost; aggregating multi-dimensional pointwise scores into a scalar reward; pairwise tournament reward for GRPO on subjective tasks (net win rate across group rollouts); generating preference pairs for DPO/RLAIF; and normalizing scores for training stability. Use when building reward models, scoring rollouts for GRPO/REINFORCE, generating preference data for DPO, or doing Best-of-N selection.
日本語の概要は準備中です。原文の説明を表示しています。
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
日本語の概要は準備中です。原文の説明を表示しています。