「judge」の検索結果

Autonomous optimization loop — hill-climb any target. Code with metrics, or skills/prompts/agents with LLM-as-judge. USE WHEN optimize, hill climb, improve metric, reduce latency, optimize skill, optimize prompt, eval mode.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias mitigation, confidence scoring, and automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the harness, scaffold, workflow, or optimizer itself is the optimization target: recursive self-improvement (RSI) loops, meta-harnesses, self-improving harnesses that mine their own failures and propose bounded edits, evolutionary or population-based search over agent scaffolds, acceptance gates for self-modifying systems, and agentic context evolution where the mechanism that produces context is versioned and evolved. Route governance of a single autonomous loop (locked surfaces, durable logs, rollback, novelty gates, approval boundaries) to harness-engineering, measurement and quality-gate design to evaluation, judge design to advanced-evaluation, and remote sandbox infrastructure to hosted-agents.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when writing, enhancing, or evaluating the launch prompt for a long-running autonomous agent or a parallel multi-agent orchestration attacking a hard problem: pseudo-formal task briefs that define terms and an exact success predicate linguistically, enumerate non-counting outcomes, set persistence rules with explicit stop and return conditions and effort floors, manage a diverse portfolio of parallel approaches with an approach registry and blocked-route bookkeeping, and gate the return on adversarial audit. Route agent topology and coordination protocols to multi-agent-patterns, runtime control surfaces and loop governance to harness-engineering, evaluator and quality-gate construction to evaluation, judge design to advanced-evaluation, and compaction or memory mechanics to context-compression and memory-systems.

日本語の概要は準備中です。原文の説明を表示しています。

Use when experiments complete to judge what claims the results support, what they don't, and what evidence is still missing. A secondary Codex agent evaluates results against intended claims and routes to next action (pivot, supplement, or confirm). Use after experiments finish — before writing the paper or running ablations.

日本語の概要は準備中です。原文の説明を表示しています。

Use when experiments complete to judge what claims the results support, what they don't, and what evidence is still missing. Codex MCP evaluates results against intended claims and routes to next action (pivot, supplement, or confirm). Use after experiments finish — before writing the paper or running ablations.

日本語の概要は準備中です。原文の説明を表示しています。

Triage a GitHub issue into something actionable — reproduce the claim, find duplicates, judge severity, and apply labels, milestone, and assignee. Use when the user asks to triage an issue, sort the backlog, deal with new issues, check whether a bug is already reported, or says "what should we do with this issue".

日本語の概要は準備中です。原文の説明を表示しています。

Periodically self-assess output quality to catch degradation before the judge does.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate a repo agent harness (AGENTS.md, rules, skills, skill refs) for broken paths/commands, redundant instructions, and usefulness using a stack-agnostic dual-judge protocol with planted traps. HIGH PRIORITY questionnaires at top: Q1 optional docs, Q2 B/C budget before Track A (certainty/tokens). A always runs after Q2; B/C opt-in. ADRs/RFCs excluded from T2. Mixed apply uses 11-mixed-apply.md (KEEP/CUT). Use when the user says harness eval, harness-eval, harness debug, audit AGENTS.md, audit skills/rules, instruction audit, redundancy of agent instructions, usefulness of skills, Ship/Review/Hold/Slim/Keep-core for harness, or wants Track A/B/C harness evaluation. Do NOT use for harness setup or init, feature spec-driven work (tlc-spec-driven), or applying Ship/Slim trims unless the user explicitly asks after the report.

日本語の概要は準備中です。原文の説明を表示しています。

Score work out of 10 against a clear rubric, then improve it round by round until every item reaches the target score the user named (for example "get it to 8 out of 10", "bring everything under 6 up to 6+"). Covers anchored rubrics, measurable criteria, independent or blind judging, before/after scorecards, and honest reporting of anything still short. Use whenever the user asks to score, rate, grade or judge something out of 10, names a target score, or asks "what would make this a 9?".

日本語の概要は準備中です。原文の説明を表示しています。

Darwin Skill 2.0 (达尔文.skill 2.0): autonomous skill optimizer, v2.0 integrates Microsoft Research SkillLens (arXiv 2605.23899) 9-dim rubric + SkillOpt (arXiv 2605.23904) validation-gated design + human-in-the-loop checkpoints. Evaluates SKILL.md files using a 9-dimension rubric (structure + effectiveness + meta-skill blacklists), runs hill-climbing with git version control, spawns independent judge agents for blind evaluation, validates improvements through test prompts with auto-break on diminishing returns, and generates visual result cards. Use when user mentions "优化skill", "skill评分", "自动优化", "auto optimize", "skill质量检查", "达尔文", "darwin", "帮我改改skill", "skill怎么样", "提升skill质量", "skill review", "skill打分".

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate staged changes using LLM-as-a-Judge before committing

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).

日本語の概要は準備中です。原文の説明を表示しています。

Make a custom AI agent — Python or TypeScript/JavaScript, on a framework or hand-built — report what it did to Failproof AI, and run your own evaluator worker (the "eval pod") that scores those runs. Reach for it on vague phrasing too: "add observability to my agent", "why isn't my agent showing up?", "run an LLM judge on our own infra". Trigger when the user wants to: • plan an integration — which points in the agent loop to record; • instrument — add `failproofai-sdk` (Python) or `@failproofai/sdk` (Node, Bun, Deno, Next.js): turn on an adapter (LangChain/LangGraph, CrewAI, LlamaIndex, Pydantic AI, Vercel AI SDK, Mastra) or wire a hand-built loop; • verify — confirm events are written, or debug an integration that produces nothing; • evaluate — write, deploy or debug an Evaluator worker in Python or TypeScript. NOT for reading telemetry or scores that already landed (that's `fp-cloud-cli`), or deciding what is worth evaluating (that's `failproofai-eval-brainstorm`).

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate Agent Skill design quality against official specifications and best practices. Use when reviewing, auditing, or improving SKILL.md files and skill packages. Provides multi-dimensional scoring and actionable improvement suggestions.

日本語の概要は準備中です。原文の説明を表示しています。

Judge and reshape the story told by an entire paper figure deck. Use when writing or revising a paper to derive a grounded brief from the manuscript and captions, review the full deck as a handling editor, and hand an ordered figure arc to `figure-composer`.

日本語の概要は準備中です。原文の説明を表示しています。

Use when asked to compare, cross-review, merge, judge, choose, or arbitrate competing plans from multiple agents such as Codex and Claude Code; when given two or more proposed plans, session IDs, transcripts, plan documents, PR descriptions, or pasted strategies; or when the user wants one recommended execution plan after agents review each other's proposals.

日本語の概要は準備中です。原文の説明を表示しています。

Records an architecture decision as a Nygard/MADR-shaped ADR under decisions/ — capturing the context that forced the choice, the options weighed, the decision itself, and its consequences in both directions, plus the supersedes chain that keeps a decision log honest. Read when asked to record an architecture decision, write an ADR, log the decision we made, document why we chose X over Y, capture this decision for the record, or supersede an old decision with a new one. Do NOT read to frame a proposal or explore an idea not yet decided (frame-a-proposal), to write a spec or implementation plan (write-a-spec), to write an incident postmortem (write-a-postmortem), or to judge whether a design is sound (review-a-design). This skill records a decision already made; it does not make one.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user wants to evaluate a creator, influencer, or brand audience using public profile signals, TikTok audience demographics, follower/following data, comments, geography, language, and content fit. Helps judge sponsorship and market fit.

日本語の概要は準備中です。原文の説明を表示しています。

Authoring and running behavioral evals for a shopping or merchant agent, covering the case shape, authoring rules, code graders and judges, the run pattern, and poisoned fixtures. Load when writing eval cases or rubrics or deciding how a suite runs.

日本語の概要は準備中です。原文の説明を表示しています。

Validates Terminal User Interface (TUI) output using freeze for screenshot capture and LLM-as-judge for semantic validation. Supports both visual (PNG/SVG) and text-based validation modes.

日本語の概要は準備中です。原文の説明を表示しています。

Judge whether a market FAILED to react to news favorable to a crowded speculative position — step 2 of Jason Shapiro's COT contrarian process. Consumes a cot-contrarian-detector report (or an explicit direction) plus a Claude-curated events JSON, fetches the underlying price series with a documented fallback chain, and produces a fail-closed CONFIRMED / NOT_CONFIRMED / INSUFFICIENT_EVIDENCE verdict using a statistically validated drift-significance test (not a naive failure-ratio, which false-confirms on pure noise). Generic beyond COT — reusable for PEAD and macro-crowding news-failure checks. Use when the user asks to check news-failure confirmation, whether a crowded market "shrugged off" good/bad news, or wants to run Shapiro step 2 on a CROWDED_LONG/CROWDED_SHORT market.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user asks to "check our positioning against what we can actually ship", "trace which differentiators we can defend", or "reconcile the positioning canvas with the claims ledger"; reconciles the reused positioning canvas against the shippable stage and the claims ledger to produce a differentiation truth set — every differentiating claim verifiable or marked '[needs source]' — that TALE-T1 is judged against. Not for building the canvas — use positioning-mapper; not for adjudicating claims — use offer-claims-registry; not for authoring the message house — use message-system-architect. 定位真相/差异化校准/可交付现实/主张核对

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate Agent Skill design quality against official specifications and best practices. Use when reviewing, auditing, or improving SKILL.md files and skill packages. Provides multi-dimensional scoring and actionable improvement suggestions.

日本語の概要は準備中です。原文の説明を表示しています。