「evals」の検索結果

Smithy the Toolsmith builds, converts, reviews, packages, validates and migrates BMad skills, agents and modules. Use when the user wants a skill, agent, persona, command or module made, changed, reviewed or packaged; a prompt or rule from another tool turned into a skill; a skill mined from session logs; an old module migrated; or asks for Smithy. Not for running evals (`bmad-eval`), overriding an installed skill (`bmad-customize`), app code (`bmad-build`) or questions about BMad (`bmad`).

日本語の概要は準備中です。原文の説明を表示しています。

LLMOps Architect: Run LLM features in production: model routing, evals, observability, caching, cost, safety.

日本語の概要は準備中です。原文の説明を表示しています。

RAG & Knowledge Architect: Retrieval systems: ingestion, chunking, embeddings, hybrid search, reranking, evals.

日本語の概要は準備中です。原文の説明を表示しています。

AI Agent Architect: Design agent systems: single vs multi-agent, tools and MCP, memory, guardrails, evals.

日本語の概要は準備中です。原文の説明を表示しています。

Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns. Use when asked to "build an eval harness", "set up eval-as-code", or "version our eval datasets".

日本語の概要は準備中です。原文の説明を表示しています。

Build automated regression suites — golden sets, threshold alerting, CI integration for model changes. Use when asked to "catch model regressions in CI", "build a golden set", or "automate eval regression".

日本語の概要は準備中です。原文の説明を表示しています。

Design an LLM eval — task schema, scoring rubric, dataset composition, and pass/fail thresholds. Use when asked to "design an LLM eval", "write a scoring rubric", or "how do we measure this model".

日本語の概要は準備中です。原文の説明を表示しています。

Analyze LLM eval results — score breakdowns by category, regression detection vs baseline, failure clustering. Use when asked to "analyze our eval results", "did the model regress", or "cluster the eval failures".

日本語の概要は準備中です。原文の説明を表示しています。

ML/AI engineer — LLM integrations, prompt engineering, model pipelines, evals, RAG. Use when asked to "add AI to this", "integrate an LLM", "build a RAG pipeline", "evaluate model accuracy", or "improve this prompt".

日本語の概要は準備中です。原文の説明を表示しています。

Patterns for continuous autonomous agent loops with quality gates, evals, and recovery controls.

日本語の概要は準備中です。原文の説明を表示しています。

Design and implement evaluation frameworks for AI agents. Use when testing agent reasoning quality, building graders, doing error analysis, or establishing regression protection. Framework-agnostic concepts that apply to any SDK.

日本語の概要は準備中です。原文の説明を表示しています。

Use when you want to benchmark and improve an existing repo skill through repeated evals, prompt mutations, and score tracking.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Use Arcade MCP to create custom MCP servers and tools with OAuth-aware authorization, evals, and deployment paths for agent tool-calling workflows.

日本語の概要は準備中です。原文の説明を表示しています。

Respan for LLM observability. Use to set up tracing or the gateway, manage prompts (create/version/deploy), run evals, experiments, and datasets, or work with traces and logs. Trigger for any Respan platform task or when instrumenting an app to send telemetry to Respan.

日本語の概要は準備中です。原文の説明を表示しています。

Nature-style remote-sensing manuscript reviewer skill for evaluating Earth-observation products, retrievals, machine-learning maps, validation design, uncertainty propagation, spatial transferability and claim-scope calibration.

日本語の概要は準備中です。原文の説明を表示しています。

Intercept, inspect, and modify HTTP/HTTPS traffic via a Rust MITM proxy with Lua scripting hooks. Supports API debugging, LLM call capture for evals, endpoint mocking, CORS/auth header injection, latency simulation, and telemetry blocking. Triggers on 'proxy', 'intercept traffic', 'inspect requests', 'mock API', 'capture API calls', 'MITM', 'network debug', 'HTTP inspection'.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate a local skill with the terum-skills CLI — hygiene, trigger evals and execution cases against a baseline, billed to the user's own Claude Code — after confirming the cost, and show the result as a Markdown board. Use when the user asks to eval, test, score or benchmark a skill, or to see whether an edit made it better.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

日本語の概要は準備中です。原文の説明を表示しています。

Use when analyzing an existing TypeScript or JavaScript codebase to decide where and how to introduce Inngest. Covers repository discovery, framework and package detection, finding durability gaps in HTTP handlers, webhooks, cron jobs, queues, long-running jobs, AI agents, Agent Evals, polling loops, eval loops, and side-effect-heavy code, then producing and implementing an incremental integration plan.

日本語の概要は準備中です。原文の説明を表示しています。

Use when validating, benchmarking, or monitoring LLM application performance. Keywords: RAG evaluation, LLM-as-a-judge, CI/CD gating, trajectory scoring, test suites, prompt quality.

日本語の概要は準備中です。原文の説明を表示しています。

AI specification planning frameworks. Use when a spec touches model calls, prompts, retrieval, tool calling, agentic loops, or evals. Covers approach selection, model and provider choice, structured output contracts, loop guards, budgets, failure modes, and eval design.

日本語の概要は準備中です。原文の説明を表示しています。