384 件 ・ 関連度順
概要と使いどころ
Judge REM memory relations, rewrites, verification, and clause carry by meaning. Use for REM update model calls that decide which older facts are superseded, which facts remain current, and whether a proposed rewrite preserves all surviving facts.
日本語の概要は準備中です。原文の説明を表示しています。
Judge which offered rows one nominated statement retires or replaces.
日本語の概要は準備中です。原文の説明を表示しています。
Judge how a memory changes active-task state and whether it refers to an existing task. Use for mem-claw task-lifecycle model calls.
日本語の概要は準備中です。原文の説明を表示しています。
Judge whether one stored row states a retired profile position as current.
日本語の概要は準備中です。原文の説明を表示しています。
Produce a high-confidence, CONCISE implementation / architecture / strategy PLAN that follows the OMC plan system end to end. It first runs OMC's real requirements interview (auto-chains the `omc-plan` skill: one question at a time, explore-first, Analyst consult) to produce an initial plan in `.omc/plans/`, then DEEPENS that plan with a 3-round ITERATIVE fusion loop — each round an Opus 4.8 panelist (Agent subagent) and a GPT-5.5 panelist (via codex) critique/improve it IN PARALLEL and BLIND, then Opus 4.8 (the orchestrator) judges and synthesizes one tighter plan that seeds the next round, stopping early on convergence. Writes the final dense plan back to the same `.omc/plans/<slug>.md`, then offers the OMC handoff (`/omc-plan --review` → `/team` or `/ralph`). Use for high-stakes planning, architecture decisions, strategy calls, and hard debugging — especially when the user runs /plan or asks how to approach a non-trivial build. Reuses the `fusion` skill's run_codex.sh + judge rubric. NOT for trivial tasks (single pass is enough).
日本語の概要は準備中です。原文の説明を表示しています。
Answer a hard question by fanning it out to a PANEL of models running in parallel — each answering independently with web search and bash, none seeing the others' work — then having Opus 4.8 judge every response into a structured analysis (consensus, contradictions, partial coverage, unique insights, blind spots) and write a final answer grounded in it. The panel is two independent Opus 4.8 runs (slug opus4.8-4.8), Opus 4.8 + GPT-5.5 via codex (opus4.8-gpt5.5), Opus 4.8 + Gemini 3.1 Pro via agy (opus4.8-gemini3.1pro), or all three (opus4.8-gpt5.5-gemini3.1pro). Opus always judges and writes the final answer — the pipeline can't be reversed. Runs on local CLI subscriptions (no metered API), and saves a timestamped provenance .md per run. Use this whenever the user asks to "run it through Fusion", says /fusion, wants a multi-model / panel / ensemble answer, wants a question cross-checked across models, or wants a higher-confidence answer with consensus and blind spots surfaced — even if they don't say "fusion". General-purpose: any topic (research, law, strategy, technical, personal). Best for high-stakes research, design calls, and debugging where being confidently wrong is expensive.
日本語の概要は準備中です。原文の説明を表示しています。
Builds an eval plan for an AI feature - error analysis on real traces, failure-mode ranking, code checks and binary LLM judges validated against human labels, CI tests, production monitoring, and a launch bar. Use when you ask how do we eval this, how do I test an AI feature, our LLM outputs are inconsistent, are the evals good enough, LLM-as-judge, hallucinations, or what accuracy is good enough to launch. Draws on 41 Lenny's Podcast guests incl. Hamel Husain, Shreya Shankar and Sander Schulhoff.
日本語の概要は準備中です。原文の説明を表示しています。
Think and work like an expert Information Retrieval Scientist. Use when a task calls for Information Retrieval Scientist judgment. Reasons from the Probability Ranking Principle, Saracevic's relevance layers, and Cranfield pooling through tuned BM25, SPLADE/ColBERT/dense and LLM reranking, trec_eval/ir_measures with paired topic-level tests, interleaving, and nugget-based RAG evaluation while treating unjudged-as-nonrelevant pools, position-biased clicks, LLM-judge circularity, single-vector embedding limits, and benchmark contamination as first-class failure modes.
日本語の概要は準備中です。原文の説明を表示しています。
Advanced Evaluation workflow skill. Use this skill when the user needs This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
日本語の概要は準備中です。原文の説明を表示しています。
Audits whether scores, benchmarks, LLM judges, or surveys measure what we think. Use when a benchmark is saturated, a score unreliable, or judge agreement untrustworthy.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants a cited, structured read of local documents and project knowledge. Triggers on: "analyze these docs", "scan my project for context", "read the docs folder", "summarize what's in .beagle/concepts/", "extract context from docs/", "what's in this folder", "go read everything in X and tell me what's there". Also invoked programmatically by other beagle skills (prfaq-beagle Ignition, brainstorm-beagle reference points, strategy-interview context grounding) via the companion contract. Does NOT trigger on codebase lookups ("find this function", "search the repo"), web research (use web-research), LLM-as-judge evaluation (use llm-judge), or document editing (use humanize-beagle). Produces a written scan plan, parallel-subagent findings, and a cited synthesis report on disk — never inline prose, never unsourced claims.
日本語の概要は準備中です。原文の説明を表示しています。
Use when comparing two or more code implementations against a spec or requirements doc. Triggers on "which repo is better", "compare these implementations", "evaluate both solutions", "rank these codebases", or "judge which approach wins". Also covers choosing between competing PRs or vendor submissions solving the same problem. Does NOT review a single codebase for quality — use code review skills instead. Does NOT evaluate strategy docs — use strategy-review. Requires a spec file and 2+ repo paths.
日本語の概要は準備中です。原文の説明を表示しています。
LLM-as-judge evaluation framework with 5-dimension rubric (accuracy, groundedness, coherence, completeness, helpfulness) for scoring AI-generated content quality with weighted composite scores and evidence citations
日本語の概要は準備中です。原文の説明を表示しています。
Trains the HUMAN user's critical thinking through four modes: drill (argument-analysis items with a single defensible answer, judge stance), scene (Socratic exploration of synthetic scenes or user-supplied material, no verdicts on interpretations; includes a configure track — design the information request and verification plan for a decision before any analysis), expedition (guided audit of impossible-tier problems from verified packs), and detective (a runtime-generated multi-layer case worked as an escape room, guide-and-judge stance). Use when the user wants to practice critical thinking, analyze arguments, hunt assumptions, examine bias, train reasoning, or decide what information a decision needs first. Triggers: critical thinking practice, train my thinking, drill, scene, byom, configure, information plan, what would I need to know, spot manipulation tactics, scam literacy, 批判思考練習, 話術辨識, detective, 查案, 破案, 偵探.
日本語の概要は準備中です。原文の説明を表示しています。
Walks a user through connecting the judge models this plugin can use — Codex, OpenCode, OpenRouter (or any compatible endpoint: 9router, z.ai, Moonshot), Gemini — and through picking models, including "research current models for me". Use for "multi setup", "set up multi", "configure judges/reviewers/ backends", "connect 9router", "change/pick models", "onboarding", or whenever a probe shows missing backends and the user needs to know what to do about it.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a task needs the judgment of an Administrative Law Judge, Adjudicator, or Hearing Officer — applying a mandatory sequential/structured evaluation framework in required order, weighing conflicting medical or vocational opinion evidence using specific regulatory factors, writing findings that explicitly address significant contrary evidence in the record, drafting a credibility/consistency finding tied to specific record citations, or checking whether a draft decision would survive substantial-evidence review.
日本語の概要は準備中です。原文の説明を表示しています。
Run work as an explicit event-driven state machine executed by subagents. The orchestrator is the runtime from the canonical loop — `state, effects = init(machine); execute(effects); while state is active: event = nextEvent(); state, effects = transition(machine, state, event); execute(effects); return state.output`. All routing judgment is authored ONCE, at plan time, into a machine file (states, event alphabet, transition table, budgets, final states); at runtime the orchestrator does pure table lookups and dispatches effects — it never decides, never authors, never ingests artifacts. Subagent/sync protocol is the timeboxed-iterating one: role prompts scaffolded once to disk and dispatched BY PATH, an initialiser preamble every subagent reads first, a self-populating cheatsheet, per-unit files, tiny structured returns, independent measurement gate. Use when the user says "workmachine", "run this as a state machine", "state-machine this", or when a goal has a small number of clearly named phases with explicit outcomes (built / passed / failed / blocked) and you want the routing to be auditable rather than judged on the fly. For clock-driven loops use timeboxed-iterating; for wide graphs with judged edge criteria use workgraph.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff may break system boundaries, dependency direction, or cross-service contracts — fifth judge dispatched by /review-changes alongside the four standard judges.
日本語の概要は準備中です。原文の説明を表示しています。
Verify a tested web agent's real behavior from its recorded Pi or Codex session and selected screenshots before scoring. The judge prompt provides absolute artifact paths; use targeted queries to compare actual outputs, errors, and the final answer with the recorded behavior, then return only the verdict JSON. Only for ego-bench judging—never to drive a browser or rerun the agent.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants to send the same prompt to several LLMs and compare the answers - sequentially (stopwatch, clean per-lane timing) or in parallel (true race), with optional repetitions per model, judged by the starting model across quality, correctness, completeness, instruction fidelity and latency (time is only one dimension). Triggers on /compare-race, 'model race', 'vergleiche modelle', 'gleicher prompt an mehrere modelle'.
日本語の概要は準備中です。原文の説明を表示しています。
テストケースの実行制御を行うフェーズスキル。各テストケースを qa-runner → qa-judge のパイプラインで実行し、アプリ状態管理と結果収集を担当する。ios-qa-workflow から参照される。
Investigate AI observability evaluations of both types — `hog` (deterministic code-based) and `llm_judge` (LLM-prompt-based). Find existing evaluations, inspect their configuration, run them against specific generations, query individual pass/fail results, and generate AI-powered summaries of patterns across many runs. Use when the user asks to debug why an evaluation is failing, surface common failure modes, compare results across filters, dry-run a Hog evaluator, prototype a new LLM-judge prompt, or manage the evaluation lifecycle (create, update, enable/disable, delete).
日本語の概要は準備中です。原文の説明を表示しています。
Generate the success criteria for a task or question, then review work against them. Given a task, goal, or open-ended question, decompose it into scenarios, evaluation perspectives, and fine-grained weighted YES/NO criteria using the Recursive Expansion Tree (RET) method; if work is supplied, score it criterion-by-criterion and surface what is missing or could be better. Use when asked to self-review or check your own work, judge whether a task is done well or completely, build a definition-of-done or completeness checklist, create an evaluation rubric or grading criteria, score or grade answers to a question, set up an LLM-as-judge rubric, or when the user mentions self-review, completeness check, success criteria, evaluation criteria, scoring rubric, Qworld, or the RET algorithm.
日本語の概要は準備中です。原文の説明を表示しています。
Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.
日本語の概要は準備中です。原文の説明を表示しています。