209 件 ・ 関連度順
概要と使いどころ
Write code evaluators for known failure modes with objective rules. Use when code can check the rule from a trace, with or without a reference answer. Use `write-judge-prompt` when the rule requires interpretation.
日本語の概要は準備中です。原文の説明を表示しています。
Run evaluations on Agent Skills to measure their effectiveness. USE FOR: "run skill evals", "evaluate my skill", "test skill quality", "check skill triggers", "skill compliance check", "measure skill performance", "run evals on [skill-name]", "grade skill execution". DO NOT USE FOR: writing skills (use skill-authoring), improving frontmatter (use sensei), or general testing unrelated to skills.
日本語の概要は準備中です。原文の説明を表示しています。
Eval-Driven Development (EDD) for AI workflows. pass@k metrics, capability evals, regression evals. Triggers: eval, edd, pass@k, capability, regression, benchmark.
日本語の概要は準備中です。原文の説明を表示しています。
Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine.
日本語の概要は準備中です。原文の説明を表示しています。
Scaffold a first eval suite for an agent: mine real failures into cases, write behavioural checks over traces, and generate the runner. Use when the user wants evals, regression tests for an agent, a golden set, or asks how to know a prompt/model change did not break things. Do NOT use for a single task's done-check (goal-test) or for auditing context (context-audit).
日本語の概要は準備中です。原文の説明を表示しています。
Audit an agent's context layout against the four places: system prompt, tools, history, tail. Use when the user asks to audit, review or fix their agent's context, prompt caching, token spend per turn, or CLAUDE.md / AGENTS.md layout. Read-only: reports findings and a fix list, changes nothing. Do NOT use for writing evals (evals-bootstrap) or for loop design (goal-test).
日本語の概要は準備中です。原文の説明を表示しています。
Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.
日本語の概要は準備中です。原文の説明を表示しています。
Use when evaluating a newly adapted model's accuracy/eval score. Runs evalscope eval (or the test/srt/run_eval.py fallback) against an OpenAI-compatible endpoint on datasets like gsm8k / aime, producing per-dataset reports and a score. Accuracy only — throughput/latency benchmarks belong to model-speed-benchmark. Triggers on accuracy, eval, evalscope, gsm8k, aime, run_eval — not throughput.
日本語の概要は準備中です。原文の説明を表示しています。
Run a skill's evals and report results. Use when the user wants to evaluate a skill, run evals, benchmark a skill, validate triggers, optimize a description, or grade skill outputs.
日本語の概要は準備中です。原文の説明を表示しています。
Braintrust is a hosted platform for evaluating and tracing LLM applications: it runs a task over a dataset, scores the outputs, stores each run as an experiment and logs production traces. Use when a user asks to write or run a Braintrust eval, score LLM outputs with autoevals, compare prompt or model versions as experiments, run evals in CI and fail on regressions, trace OpenAI or Anthropic calls with Braintrust, manage evaluation datasets, or use the bt CLI.
日本語の概要は準備中です。原文の説明を表示しています。
Use when the user wants to work on a Claude Code skill file (SKILL.md): writing one from scratch, testing whether an existing one works well, running evals or benchmarks, improving its instructions, or fixing why it isn't triggering. Triggers on: 'make a skill for X', 'test this skill', 'run evals on my SKILL.md', 'touch-skill', sharing a SKILL.md and asking if it's ready to ship. The key signal is intent to create, validate, or improve a skill — not just mention one. Do NOT trigger for general Claude Code questions, hook debugging, or CLAUDE.md configuration.
日本語の概要は準備中です。原文の説明を表示しています。
Builds QA harnesses and agent evals. Use when evaluating tool, trace, red-team, regression, multi-agent, or carried-workspace coding trajectories. Not for metric design (ai-evals).
日本語の概要は準備中です。原文の説明を表示しています。
Designs LLM, agent, responsible-AI, and multimodal evals. Use when measuring quality, fairness, grounding, safety, or judge reliability. Not for test harnesses (qa-agent-testing).
日本語の概要は準備中です。原文の説明を表示しています。
Audit existing eval coverage — gaps, metric validity, benchmark leakage, dataset freshness. Use when asked "what eval coverage do we have", "audit our evals", or "check for benchmark leakage".
日本語の概要は準備中です。原文の説明を表示しています。
Power Platform ソリューションの全体アーキテクチャを設計する。Copilot Studio / Power Automate / Code Apps / Copilot Managed Runtime / Power Pages / AI Builder の使い分け判断、コンポーネント選定、統合パターンを決定する。社内向け画面は利用者のライセンス（Power Apps Premium か、Copilot Credits の従量課金か）と利用頻度をヒアリングし、Code Apps と Copilot Managed Runtime を使い分ける。構成が確定したら、実装着手前に admin スキルで環境チェック（既定環境ではないか・マネージド環境・Code Apps / MCP の有効化・セキュリティ ロール）と DLP 事前チェック（使用コネクタがブロックされていないか・Business / Non-business が混在しないか）を実行して結果をユーザーに提示する。Agent 365 の AI チームメイトを採用する場合はライト実装（PoC）と本格実装（private リポジトリ + CI/CD + Agent Evals）を AskUserQuestion で選ばせ、Git ホスティング（GitHub / Azure DevOps Repos / その他）も確定してから実装へ進む。外部の文章を読むエージェントではプロンプト インジェクション対策を設計段階で工数に含める。
Design, create, run, monitor, and maintain Vapi Simulations for assistants and squads. Use for turning plain-language requirements into a coverage plan, simulation personalities, scenarios, structured-output success criteria, simulations, suites, extending an existing suite, chat or voice runs, tool mocks, target variables, lifecycle webhooks, regression coverage, launch readiness, CI quality gates, run-result analysis, and simulation API validation errors. Do not use for fixed-turn mock-conversation Evals unless the user is deciding between Evals and Simulations.
日本語の概要は準備中です。原文の説明を表示しています。
Skill Optimization: Auto Research. Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill, benchmark skill, eval my skill, run evals on. Outputs: an improved SKILL.md, a results log, and a changelog of every mutation tried.
日本語の概要は準備中です。原文の説明を表示しています。
Use when building, migrating, or debugging Agent Evals on Inngest: scoring AI agent or workflow outcomes, deferred scorers, sessions, traces, step experiments, experiment variant attribution, Insights queries, or production eval loops for prompts, models, tools, providers, and agent behavior. Covers TypeScript SDK v4 scoring beta APIs, `scoreMiddleware`, `step.score`, `inngest.score`, `createScorer`, `defer`, `group.experiment`, `experimentRef`, `meta.sessions`, and when to use durable workflow primitives for outcome-based evaluation.
日本語の概要は準備中です。原文の説明を表示しています。
Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.
日本語の概要は準備中です。原文の説明を表示しています。
Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.
日本語の概要は準備中です。原文の説明を表示しています。
Use after changing prompts, skills, the compiler, or editorial policies — or when the user asks to measure edit quality, run regression evals, check agreement with approved edits, or says '評価して', '一致率を測って', 'eval を回して'. Compares agent output against human-approved golden projects.
Inngest イベント駆動 Durable Execution プラットフォームリファレンス。 サーバーレス環境での長時間ワークフロー、バックグラウンドジョブ、スケジュールタスク実装。 createFunction, step.run, step.sleep, step.sleepUntil, step.waitForEvent, step.waitForSignal, step.invoke, step.sendEvent, step.fetch, step.ai.infer, step.ai.wrap, inngest.send, serve, connect。 concurrency, throttle, debounce, rate-limit, batching, priority, idempotency, singleton フロー制御。 Realtime（channels, topics, publish, subscribe, useRealtime, subscription tokens）による進捗配信。 Agent Evals, scoring, group.experiment, MCP dev server などエージェント運用機能。 Next.js / Express / Hono / Cloudflare Workers / Vercel / Netlify / DigitalOcean / Render デプロイ対応。
Claude API (platform.claude.com) のプロンプトエンジニアリング・評価リファレンス。 prompt engineering, XML タグ, few-shot, system prompts, モデル別プロンプティング（Fable 5 / Opus 5 / Sonnet 5）。 evals, grading（LLM-graded 等）, jailbreak・prompt leak 対策, hallucination 低減, classification, ticket routing 等ユースケース実装。
Evaluate and improve an existing Claude Code skill using explicit success criteria and small controlled changes. Use when the user asks to optimize a skill, reduce over-triggering or under-triggering, improve reliability, tighten instructions, or add evals for a skill. Also trigger on "スキルを改善して", "スキルを最適化して", "スキルの品質を確認して".