「eval」の検索結果

Design AI evaluation frameworks with benchmarks. TRIGGERS - Use when user needs help with ai-eval-framework related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate AI copilot performance. TRIGGERS - Use when user needs help with ai-copilot-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate AI agent performance and reliability. TRIGGERS - Use when user needs help with ai-agent-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate third-party logistics providers. TRIGGERS - Use when user needs help with 3pl-evaluation related tasks.

日本語の概要は準備中です。原文の説明を表示しています。

Use when building durable AI agents or agentic workflows with Inngest and AgentKit, including model calls, tool calls, multi-agent networks, human approval, realtime progress, provider rate limits, crash-safe execution, and Agent Evals handoff. Covers AgentKit, `step.ai`, `step.run`, `step.waitForEvent`, native realtime, and when to use lower-level Inngest primitives instead of an in-memory agent loop. Use `inngest-agent-evals` with this skill when the user wants scoring, sessions, experiments, deferred scorers, or outcome-based evaluation for the agent.

日本語の概要は準備中です。原文の説明を表示しています。

"Apply the TPACK framework to evaluate and design technology-integrated instruction at the intersection of technological, pedagogical, and content knowledge. Use this skill when the user needs to assess teacher readiness for technology integration, design professional development for ed-tech, or evaluate whether technology use is pedagogically grounded — even if they say 'how to integrate technology in teaching', 'ed-tech evaluation', or 'teacher technology competency'.".

日本語の概要は準備中です。原文の説明を表示しています。

AI specification planning frameworks. Use when a spec touches model calls, prompts, retrieval, tool calling, agentic loops, or evals. Covers approach selection, model and provider choice, structured output contracts, loop guards, budgets, failure modes, and eval design.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation

日本語の概要は準備中です。原文の説明を表示しています。

Add a new benchmark to sgl-eval by vendoring its NeMo-Skills dataset module and registering it in `_TABLE`. Use when the user asks to "add <benchmark>" / "support <benchmark>" / "register <benchmark>" inside the sgl-eval repo.

日本語の概要は準備中です。原文の説明を表示しています。

Marketing analytics and data-driven optimization for B2B SaaS campaigns. Use this skill when analyzing marketing performance, optimizing conversion rates, running A/B tests, conducting customer research, creating marketing dashboards, building attribution models, analyzing funnel metrics, or storytelling with data. Also triggers on: marketing ops, performance analytics, CRO, A/B testing, data storytelling, customer insights, tracking plan, event taxonomy, UTM taxonomy, GA4 audit, consent mode, instrumentation contract, sample size calculation, minimum detectable effect, test duration, not enough traffic to A/B test, statistical significance, Bayesian vs frequentist, sample ratio mismatch, experiment feasibility, martech stack, marketing tech stack audit, tool consolidation, build vs buy, vendor evaluation, shadow IT tools, ESP/CRM/platform migration, switching cost, cutover plan, suppression list migration, data portability, customer interviews, survey design, sampling frame, non-response bias, how many people should I interview, thematic saturation, voice of customer, jobs-to-be-done, buyer persona research, is this finding real, data enrichment, which enrichment provider should we buy, waterfall enrichment, contact data quality, match rate, our list data is bad, email verification, catch-all domain, accept-all, bounce rate from bad data, B2B data decay, how often should we re-verify contacts, ZoomInfo vs Apollo vs Clearbit evaluation, cost per usable record, where did this list come from, GDPR Article 14 notice, purchased list compliance, suppression screening, TAM list building, demand planning, marketing plan, how many leads do we need, what pipeline target should marketing carry, funnel model, top-down vs bottom-up plan, plan of record, annual marketing planning, quarterly planning, pipeline coverage we have to create, sales cycle lag, our leads cannot close this quarter, carry-in pipeline, scenario planning, conservative moderate aggressive forecast, stretch target, reforecast, re-forecast triggers, assumption register, capacity ceiling, addressable audience ceiling, marginal cost per lead, blended CPL is wrong, we missed plan and nobody knows why, AI policy for the marketing team, AI governance, AI acceptable use policy, which AI tools are we allowed to use, can I paste customer data into ChatGPT, shadow AI, AI use register, human review of AI output, when can we stop reviewing every AI draft, can the agent send on its own, the AI made a mistake now what, pull back AI autonomy, AI evals, our AI vendor changed the model, AI output went out wrong, prompt injection, can our AI read prospect replies safely, lethal trifecta, the AI leaked internal notes, turn off auto-generated ad assets, auto-applied recommendations, AI literacy training, EU AI Act Article 4, duplicate records in the CRM, dedupe before import, which value wins in a merge, CRM cleanup.

日本語の概要は準備中です。原文の説明を表示しています。

Build evaluation frameworks for LLM systems — harness design, graders, datasets, regression tracking, and human-in-the-loop eval. Use when you need systematic measurement of model or agent quality.

日本語の概要は準備中です。原文の説明を表示しています。

Run a Meta-Harness-style optimization loop NATIVELY — automatically search over the scaffolding around a FIXED base model (memory, retrieval, context construction, prompt templates, summarization, tool-selection logic) by proposing candidate variants, scoring each on a cheap deterministic eval, and keeping a Pareto frontier of quality vs cost — using native Agent / Workflow / loop tools instead of a standalone Python harness. Use this whenever the user wants to optimize, evolve, tune, distill, or search over a harness, scaffold, prompt system, memory or retrieval policy, context-assembly code, or summarizer while keeping the model fixed; whenever they mention Meta-Harness, harness optimization, scaffold evolution, automatic prompt/memory optimization, an evolutionary or Pareto search over candidate implementations, or "make the harness/agent better without retraining"; and whenever the gain must come from the code AROUND the model rather than the model weights. Reproduces the Meta-Harness paper's method natively, with no claude_wrapper.py and no metered solver API.

日本語の概要は準備中です。原文の説明を表示しています。

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate an incoming workflow upgrade — a skill, MCP server, subagent, library, external service, prompt, or pattern — through one disciplined pipeline before adopting it. Security is a blocking gate. Records every verdict so the same thing is never evaluated twice. Use when considering a new tool from a post, repo, or recommendation. Triggers — "vet this", "should we adopt", "is this safe to install", "evaluate this skill/MCP", "what do you think of this tool".

日本語の概要は準備中です。原文の説明を表示しています。

Scope, preregister, execute, audit, normalize, and publish GEODE evaluations with research-question, reproduction, attempt-lineage, trajectory, verifier-receipt, and immutable artifact contracts. Use for benchmark runs or comparisons, eval-artifact and cache-accounting audits, GPT or subscription result normalization, research question or hypothesis framing, retry diagnosis, trajectory publication, DPO or reward-data readiness reviews, and requests involving MCPMark, tau2, BFCL, Agent-World, HAL, Terminal-Bench, or Toolathlon.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate Jira ticket complexity and inconsistencies via jira-evaluator agent, read-only. NOT estimation (mk:jira-estimator) or full RCA (mk:jira-analyst).

日本語の概要は準備中です。原文の説明を表示しています。

Experimental behavioral evaluator. Drives a running artifact via browser/curl/CLI and records rubric evidence; the runner is not yet a fully implemented automated evaluation system.

日本語の概要は準備中です。原文の説明を表示しています。

Use when loading, listing, composing, or validating graded evaluation rubrics — PASS/WARN/FAIL grading with weighted criteria and balanced PASS/FAIL anchor examples. Triggers on /mk:rubric, "load rubric", "compose rubric preset", "validate rubric", or evaluator-style code review that needs anchor-grounded judgments.

日本語の概要は準備中です。原文の説明を表示しています。

Use when a decision needs deep multi-source technical research with cited evidence — technology evaluation, ecosystem comparison, standards/spec fact-finding, "how do others solve X". Triggers on "research X", "deep dive on X", "evaluate X vs Y", "find best practices for X". NOT for library/API doc lookup (see mk:docs-finder); NOT for one-shot URL fetch (see mk:web-to-markdown); NOT for internal codebase discovery (see mk:scout); NOT for project-only Q&A (see mk:ask-me); NOT for comparing solution designs (see mk:brainstorming); NOT for root-cause debugging (see mk:investigate); NOT for wiki ingestion of sources (see mk:wiki-research).

日本語の概要は準備中です。原文の説明を表示しています。

Audit whether a declared local review evaluation can support fair quality claims. Use only when the user explicitly asks to audit review-eval validity or use the bundled synthetic fixture.

日本語の概要は準備中です。原文の説明を表示しています。

Data structures and algorithms for AI agent episodic memory. Covers vector stores (HNSW, IVF, PQ), temporal indexing, knowledge graphs with triple stores, hierarchical summarization, forgetting curves, working/long-term/ procedural memory, and memory consolidation. Deep analysis of MemGPT/Letta, Zep/Graphiti, Mem0, and the Stanford generative agents memory architecture. Teaches the CS fundamentals behind how agents remember, retrieve, and forget. Activate on: "agent memory", "episodic memory", "vector search algorithm", "HNSW", "memory retrieval", "forgetting curve", "knowledge graph memory", "MemGPT", "Letta", "Zep", "Mem0", "memory consolidation", "temporal retrieval", "agent long-term memory", "memory layer". NOT for: conversation protocol design (use agent-conversation-protocols), agent infrastructure selection (use agentic-infrastructure-2026), building RAG pipelines (use ai-engineer).

日本語の概要は準備中です。原文の説明を表示しています。

Autonomously improve a real artifact (code, training recipe, agent harness, data pipeline, prompt) against an objective and an evaluator, using Hypothesis Tree Refinement (HTR) from the Arbor paper. Use this whenever someone wants to iteratively optimize something over many experiments without overfitting — e.g. "get my model's eval score up", "improve this agent/harness", "tune this pipeline", "beat the baseline on this benchmark", "run a search over approaches and keep the best", "do an MLE-bench / Kaggle-style optimization", or any long-horizon "make this artifact better and don't just memorize the dev set" task. Trigger it even when the user doesn't say "Arbor" or "hypothesis tree" but describes repeated experiment-and-evaluate loops, branching exploration of competing ideas, or worries about a dev/test gap. Runs Claude itself as the coordinator with subagent executors in isolated git worktrees; for the standalone `arbor` CLI tool see references/arbor-upstream.md.

日本語の概要は準備中です。原文の説明を表示しています。

This skill recommends re-evaluating the need for treatment of other pituitary hormone deficiencies in the postoperative period following surgical intervention for Cushing's syndrome. It is triggered when assessing a post-op CS patient for ongoing pituitary function, evaluating for hypopituitarism, or checking HPA axis recovery with morning cortisol/ACTH stimulation tests and pituitary MRI within 1–3 months.

日本語の概要は準備中です。原文の説明を表示しています。

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.

日本語の概要は準備中です。原文の説明を表示しています。