「evals」の検索結果

Design a self-improvement loop for a shipped product's AI agents — production traces feed evals, evals feed improvement proposals (prompts, playbooks, retrieval configs), and a human approval gate promotes changes with rollback. Technique-agnostic: chooses per project between ACE-style evolving playbooks, GEPA/MIPROv2 offline optimization, or simple eval-driven iteration via references/techniques.md. Load when the user asks to make my product's agents self-improving, learn from production traces, add a learning loop, evolve prompts or playbooks safely, promote agent improvements, or GEPA-style optimization. NOT harness-evolution (that improves the coding agent), NOT experimentation (product A/B tests), NOT agent-run-retro (dev-phase manual retros — this skill is the production-scale continuation).

日本語の概要は準備中です。原文の説明を表示しています。

AIによる開発の成功条件を実装前に定義し、コード・ルール・モデル・人の評価で新機能と既存機能を確認しながら、複数回の試行から信頼性を記録するスキル。

• AI実装前に成功条件を決めたいとき
• プロンプトやエージェント変更後の回帰確認
• 複数回の試行で信頼性を測りたいとき

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Use for Codex models/pricing, scheduled tasks, skills, settings, setup, troubleshooting, customization, automations, and self-knowledge—including 'you,' 'your,' 'this app,' or 'this coding agent' when they refer to Codex—and for OpenAI APIs/products and ChatGPT Work. Also use for model choice/migration, prompting, SDKs, Responses, Realtime, agents, evals, and Chat/Work/Codex comparisons. Do not use for generic app/software tasks that merely mention Codex.

日本語の概要は準備中です。原文の説明を表示しています。

Choose, inspect, validate, and report Paperclip Runner or Product E2E evaluations while preserving evidence, provenance, cost, and failure classification.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Use for Codex models/pricing, scheduled tasks, skills, settings, setup, troubleshooting, customization, automations, and self-knowledge—including 'you,' 'your,' 'this app,' or 'this coding agent' when they refer to Codex—and for OpenAI APIs/products and ChatGPT Work. Also use for model choice/migration, prompting, SDKs, Responses, Realtime, agents, evals, and Chat/Work/Codex comparisons. Do not use for generic app/software tasks that merely mention Codex.

日本語の概要は準備中です。原文の説明を表示しています。

Runs the evals of a BMad skill through the agent harness the user works in and reports what they show: baseline against the bare model, variant against a stripped or prior version, quality against a rubric, and trigger accuracy of the description with an optional optimization loop. Use when the user asks to evaluate, benchmark, test the triggers of, grade, or compare versions of a skill, or when the bmad-toolsmith skill asks for an eval of a skill it built.

日本語の概要は準備中です。原文の説明を表示しています。

Smithy the Toolsmith builds, converts, reviews, packages, validates and migrates BMad skills, agents and modules. Use when the user wants a skill, agent, persona, command or module made, changed, reviewed or packaged; a prompt or rule from another tool turned into a skill; a skill mined from session logs; an old module migrated; or asks for Smithy. Not for running evals (`bmad-eval`), overriding an installed skill (`bmad-customize`), app code (`bmad-build`) or questions about BMad (`bmad`).

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.

日本語の概要は準備中です。原文の説明を表示しています。

Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing traces or spans, investigating LLM/agent failures, deciding what to do after instrumenting an app, building failure taxonomies, choosing what evals to write, or asking "what's going wrong", "what kinds of mistakes", or "where do I focus" — even without naming a technique.

日本語の概要は準備中です。原文の説明を表示しています。

Build and run evaluators for AI/LLM applications using Phoenix.

日本語の概要は準備中です。原文の説明を表示しています。

Improve AI application with evaluation-driven development. Define eval criteria, instrument the application, build golden datasets, observe and evaluate application runs, analyze results, and produce a concrete action plan for improvements. ALWAYS USE THIS SKILL when the user asks to set up QA, add tests, add evals, evaluate, benchmark, fix wrong behaviors, improve quality, or do quality assurance for any Python project that calls an LLM model.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

LLM Operations -- RAG, embeddings, vector databases, fine-tuning, prompt engineering avancado, custos de LLM, evals de qualidade e arquiteturas de IA para producao.

日本語の概要は準備中です。原文の説明を表示しています。

Skill validation framework PLUS daily test-suite health and regression intelligence. Validates skill conformance (frontmatter, manifest coverage, resolver coverage). Runs the project test suite in tiered phases (unit / evals / integration / system health), classifies failures, and produces a regression-aware report.

日本語の概要は準備中です。原文の説明を表示しています。

Build durable backend AI agents with the eve framework. Use when creating, editing, or debugging an eve project — agent instructions, skills, tools, connections, channels, sandboxes, subagents, schedules, or evals.

日本語の概要は準備中です。原文の説明を表示しています。

Write or extend an E2E journey spec in evals/specs that proves a PR's change to a human reviewer. Use when a PR changes user-visible behaviour, when a reviewer asks "show me", or when the coverage decision says a journey is missing.

日本語の概要は準備中です。原文の説明を表示しています。

Evolutionary explanation-discovery engine for hard-to-vary causal theories of unknown phenomena. USE WHEN develop a new theory, explain a mystery, solve an explanatory gap, discover a mechanism, consciousness, origin, or scientific breakthrough. Also use for multi-cycle creative exploration. NOT FOR a single-pass divergent brainstorm (use BeCreative) or delivery validation alone (use Evals).

日本語の概要は準備中です。原文の説明を表示しています。

Divergent ideation and corpus expansion via Verbalized Sampling plus extended thinking — single-shot generates several internally diverse candidates and surfaces the strongest, or expands a seed corpus into a diverse dataset for evals and test sets. USE WHEN be creative, brainstorm, divergent ideas, creative solutions, maximum creativity, tree of thoughts, radically different, name this, creative angle, expand this corpus, synthetic data, generate diverse examples, create test set. NOT FOR multi-cycle evolutionary ideation with Lamarckian meta-learning (use Ideate).

日本語の概要は準備中です。原文の説明を表示しています。

Hardens LifeOS tests via property/mutation testing. USE WHEN harden, hardening, property test, property based testing, PBT, fast-check, mutation test, mutation testing, Stryker, CRAP score, CRAP analysis, DRY scan, jscpd, acceptance test mutation, strengthen tests, sharpen ISCs, find bugs example tests miss, universal quantified claim, shrink counterexample, what bugs am I missing, test the tests, test of the test. NOT FOR writing new feature tests (use bun test directly), grading agent output quality (use Evals), UI verification with real Chrome (use Interceptor), finding security vulnerabilities, or building new functionality.

日本語の概要は準備中です。原文の説明を表示しています。

Instrument an AI application with DeepEval's native tracing so its behavior is visible in Confident AI. TRIGGER when the user wants to add DeepEval tracing or @observe to an LLM app, agent, RAG pipeline, or chatbot; wire a framework, model-provider, or vector-database integration (LangGraph, LangChain, OpenAI Agents, LlamaIndex, Pydantic AI, CrewAI, and others); choose between a native integration and manual instrumentation; set span types, tags, or metadata; or send DeepEval-SDK traces to Confident AI's Observatory. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the `deepeval` skill), or for raw OpenTelemetry / OTLP export without the deepeval package (use the `deepeval-otel` skill). This skill is purely DeepEval-SDK instrumentation — producing well-formed traces, not running evals.

日本語の概要は準備中です。原文の説明を表示しています。