「llm-as-judge」の検索結果

Run AI agent and LLM evaluations in CI/CD pipelines — automated quality gates that fail the build when AI output quality drops. Use when someone asks to "test my AI agent", "add evals to CI", "catch prompt regressions", "compare models", "evaluate LLM output quality", "set up AI quality gates", or "benchmark my agent before deploying". Covers eval frameworks (Cobalt, Promptfoo, Braintrust), LLM-as-judge scoring, threshold-based assertions, and GitHub Actions integration.

日本語の概要は準備中です。原文の説明を表示しています。

Detect AI-generated text using rule-based analysis, LLM-as-judge scoring, and optional external APIs. Use when: auditing content for AI authorship, academic integrity checks, editorial review, SEO content audits.

日本語の概要は準備中です。原文の説明を表示しています。

Advanced Evaluation workflow skill. Use this skill when the user needs This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

日本語の概要は準備中です。原文の説明を表示しています。

Design well-structured agent loops with best-practice coaching and cross-model review gates before you run them. Use when the user wants to design, build, or set up an agent loop, iterative agent workflow, self-review loop, LLM-as-judge loop, multi-model council, reviewer/judge gate, or goal-driven looping process. Guides goal refinement, typed verification criteria, reviewer/judge selection, privacy boundaries, termination guards, and observability, then emits a RUN_IN_SESSION.md handoff prompt plus portable loop.yaml, loop.resolved.json, LOOP.md, and run-loop.py.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user wants a cited, structured read of local documents and project knowledge. Triggers on: "analyze these docs", "scan my project for context", "read the docs folder", "summarize what's in .beagle/concepts/", "extract context from docs/", "what's in this folder", "go read everything in X and tell me what's there". Also invoked programmatically by other beagle skills (prfaq-beagle Ignition, brainstorm-beagle reference points, strategy-interview context grounding) via the companion contract. Does NOT trigger on codebase lookups ("find this function", "search the repo"), web research (use web-research), LLM-as-judge evaluation (use llm-judge), or document editing (use humanize-beagle). Produces a written scan plan, parallel-subagent findings, and a cited synthesis report on disk — never inline prose, never unsourced claims.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user wants web research: gathering cited, multi-angle evidence on a specific question. Triggers on: "research X for me", "do web research on", "look up sources for", "find citations for", "gather evidence on", "what does the web say about X". Also invoked programmatically by other beagle skills (prfaq-beagle Ignition, brainstorm-beagle reference points, strategy-interview context grounding) via the companion contract. Does NOT trigger on codebase lookups ("find this function", "search the repo"), local file search, LLM-as-judge evaluation, or paywalled/auth-gated scraping. Produces a written plan, parallel-subagent findings, and a cited synthesis report on disk — never inline prose, never unsourced claims.

日本語の概要は準備中です。原文の説明を表示しています。

模型评估全流程。覆盖分类/回归/排序/生成/RAG/LLM 评估、A/B 测试、 在线监控、数据漂移检测、评估集构建、基准测试。 触发关键词：评估、指标、Precision、Recall、F1、AUC、BLEU、ROUGE、 A/B 测试、基准测试、数据漂移、模型监控、RAGAS、LLM-as-Judge。

日本語の概要は準備中です。原文の説明を表示しています。

LLM-as-judge evaluation framework with 5-dimension rubric (accuracy, groundedness, coherence, completeness, helpfulness) for scoring AI-generated content quality with weighted composite scores and evidence citations

日本語の概要は準備中です。原文の説明を表示しています。

Master context engineering for AI agent systems. Use when designing agent architectures, debugging context failures, optimizing token usage, implementing memory systems, building multi-agent coordination, evaluating agent performance, or developing LLM-powered pipelines. Covers context fundamentals, degradation patterns, optimization techniques (compaction, masking, caching), compression strategies, memory architectures, multi-agent patterns, LLM-as-Judge evaluation, tool design, and project development.

日本語の概要は準備中です。原文の説明を表示しています。

Set up LLM-as-judge evaluation for ORC workflows

日本語の概要は準備中です。原文の説明を表示しています。

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise compari

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

Generate the success criteria for a task or question, then review work against them. Given a task, goal, or open-ended question, decompose it into scenarios, evaluation perspectives, and fine-grained weighted YES/NO criteria using the Recursive Expansion Tree (RET) method; if work is supplied, score it criterion-by-criterion and surface what is missing or could be better. Use when asked to self-review or check your own work, judge whether a task is done well or completely, build a definition-of-done or completeness checklist, create an evaluation rubric or grading criteria, score or grade answers to a question, set up an LLM-as-judge rubric, or when the user mentions self-review, completeness check, success criteria, evaluation criteria, scoring rubric, Qworld, or the RET algorithm.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

日本語の概要は準備中です。原文の説明を表示しています。

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

日本語の概要は準備中です。原文の説明を表示しています。

Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

日本語の概要は準備中です。原文の説明を表示しています。

INVOKE THIS SKILL for LLM-as-judge evaluation workflows on Arize: creating/updating evaluators, running evaluations on spans or experiments, tasks, trigger-run, column mapping, and continuous monitoring. Use when the user says: create an evaluator, LLM judge, hallucination/faithfulness/correctness/relevance, run eval, score my spans or experiment, ax tasks, trigger-run, trigger eval, column mapping, continuous monitoring, query filter for evals, evaluator version, or improve an evaluator prompt.

日本語の概要は準備中です。原文の説明を表示しています。