「eval」の検索結果

Curate Busabase training examples, create immutable dataset snapshots, run reproducible LoRA or QLoRA fine-tunes on a local Apple Silicon Mac with MLX-LM, compare the untouched baseline and adapter on locked evaluations, and register approved adapters. Use when the user wants to train, fine-tune, evaluate, compare, promote, or manage a local language model with Busabase as the workflow source of truth. Do not use for RAG-only knowledge retrieval, cloud training, or training a foundation model from scratch.

日本語の概要は準備中です。原文の説明を表示しています。

Review board (Busabase App-in-Skill) that runs a fixed suite of mock test cases against a baseline vs candidate agent version and surfaces rubric-scored regressions before a release. Use when the user invokes $kelly-agent-eval or /kelly-agent-eval, wants to review agent-version regressions, compare baseline vs candidate quality, triage a release, or record a release approve/block decision. Deterministic mock rubric scores only — not a real LLM-judge call, and it never deploys anything.

日本語の概要は準備中です。原文の説明を表示しています。

Focused Signals scout for PostHog projects using feature flags. Watches the flag roster and the `$feature_flag_called` evaluation stream for contradictions between a flag's configured state and its real traffic: evaluation cliffs on healthy flags, ghost flags (code calling keys that no longer exist), response-distribution shifts with no corresponding flag edit, and flag debt (stale, fully-rolled-out, or dead flags still burning evaluations). Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other skills.

日本語の概要は準備中です。原文の説明を表示しています。

Rigorous methodology for evaluating ML models on established benchmarks. Covers proper train/val/test splits, baseline verification from original papers, exact metric formula discrepancies, data-leak detection checklist, multi-seed robustness, and honest reporting templates. Use when claiming to beat published baselines, writing methods papers, or auditing existing results.

日本語の概要は準備中です。原文の説明を表示しています。

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

日本語の概要は準備中です。原文の説明を表示しています。

Generate the success criteria for a task or question, then review work against them. Given a task, goal, or open-ended question, decompose it into scenarios, evaluation perspectives, and fine-grained weighted YES/NO criteria using the Recursive Expansion Tree (RET) method; if work is supplied, score it criterion-by-criterion and surface what is missing or could be better. Use when asked to self-review or check your own work, judge whether a task is done well or completely, build a definition-of-done or completeness checklist, create an evaluation rubric or grading criteria, score or grade answers to a question, set up an LLM-as-judge rubric, or when the user mentions self-review, completeness check, success criteria, evaluation criteria, scoring rubric, Qworld, or the RET algorithm.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate AI model performance with metrics, benchmarks, and comparison. TRIGGERS - Use when user needs help with ai-model-evaluator related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Design AI evaluation frameworks with benchmarks. TRIGGERS - Use when user needs help with ai-eval-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Create evaluation frameworks for AI outputs with rubrics and scoring. TRIGGERS - Use when user needs help with ai-evaluation-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate AI copilot performance. TRIGGERS - Use when user needs help with ai-copilot-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate third-party logistics providers. TRIGGERS - Use when user needs help with 3pl-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Claude API (platform.claude.com) のプロンプトエンジニアリング・評価リファレンス。 prompt engineering, XML タグ, few-shot, system prompts, モデル別プロンプティング（Fable 5 / Opus 5 / Sonnet 5）。 evals, grading（LLM-graded 等）, jailbreak・prompt leak 対策, hallucination 低減, classification, ticket routing 等ユースケース実装。

Uncorrelated Evaluator — 4軸ルーブリック専門エージェント（functionality-reviewer・code-reviewer・design-reviewer・originality-reviewer）と security-reviewer を5並列実行し、成果物を多面的に独立評価します。PASS / NEEDS WORK / BLOCKED 判定を返します。Self-leniency 防止のため、実装エージェントのコンテキストを引き継がない新規エージェントとして起動します。「レビューして」「評価して」「実装を確認して」などのリクエストで起動します。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Claude Codeセッションの効率を分析・スコアリングするWebツール。セッションログ（JSONL）を解析し、7観点100点満点で「無駄の有無」を評価。npx session-evaluatorで起動し、ブラウザUIでセッション選択→分析→改善提案まで実行可能。

Evaluate any project, product, or system's claims against its actual implementation — scoring each claim for truth, identifying architectural gaps, assessing competitive positioning, proposing creative solutions, and producing an actionable roadmap. Load when the user asks to evaluate claims, reality-check a project, assess what this project actually does vs what it says, validate product claims, or score a system's credibility. Also triggers on "is this real", "does this work as claimed", "evaluate this project", "assess the gap between claims and reality", "how credible is this", "investor assessment", "score these claims", or "what's real vs marketing".

日本語の概要は準備中です。原文の説明を表示しています。

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Execute a bounded Coval evaluation on explicit test cases, then audit conversation, recording and metric evidence. Use when the user requests a run; does not automatically rerun, tune an agent or schedule evaluations.

日本語の概要は準備中です。原文の説明を表示しています。

Audit an existing Coval evaluation setup for coverage, misleading scores, missing evidence, judge validation and execution risk. Read-only; use when inheriting an eval or deciding whether its results support a release decision.

日本語の概要は準備中です。原文の説明を表示しています。

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

日本語の概要は準備中です。原文の説明を表示しています。