「eval」の検索結果

Authoring and running behavioral evals for a shopping or merchant agent, covering the case shape, authoring rules, code graders and judges, the run pattern, and poisoned fixtures. Load when writing eval cases or rubrics or deciding how a suite runs.

日本語の概要は準備中です。原文の説明を表示しています。

Azure Resource Manager SDK for Arize AI Observability and Evaluation (.NET). Use when managing Arize AI organizations on Azure via Azure Marketplace, creating/updating/deleting Arize resources, or integrating Arize ML observability into .NET applications. Triggers: "Arize AI", "ML observability", "ArizeAIObservabilityEval", "Arize organization".

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate Expo skills in this repo end-to-end - trigger accuracy, generated code quality, and runtime screenshots on iOS simulator and Android emulator via Expo Go (web optional). Use when the user wants to eval an Expo skill, test that a skill produces working code, benchmark a skill with device screenshots, or verify a skill's output renders correctly.

日本語の概要は準備中です。原文の説明を表示しています。

适用于 Claude Code 会话的正规评测框架（Evaluation Framework），实现了评测驱动开发（Eval-Driven Development, EDD）原则

日本語の概要は準備中です。原文の説明を表示しています。

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

日本語の概要は準備中です。原文の説明を表示しています。

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

日本語の概要は準備中です。原文の説明を表示しています。

Scaffold a new brooks-lint analysis skill so it passes `npm run validate` and `npm run evals` on the first try — generates skills/{name}/SKILL.md (with the mandatory "Do NOT trigger for:" clause and a Process section citing guide step ranges) plus skills/{name}/{name}-guide.md (sequentially numbered steps), then appends paired eval scenarios. Triggers when the maintainer asks to "add a new skill", "scaffold a skill", or "create a brooks-lint mode". Do NOT trigger for: editing an existing skill's content, adding a single eval to an existing skill, or authoring skills for some other plugin.

日本語の概要は準備中です。原文の説明を表示しています。

Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against the expected outcome, and let the gaps drive the edits. Use when the user wants to test/eval/improve/harden a skill, says "this skill keeps producing X / keeps missing Y", or hands a skill plus example input→expected-output pairs. Pairs with [write-skills](../write-skills/SKILL.md) (the authoring principles every fix obeys).

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate YC batch companies for investment — scrapes the YC directory, researches each company and its founders (work history, LinkedIn, website), assesses founder-company fit, and exports to Google Sheets with priority rankings. Use when asked to evaluate YC companies, research a YC batch, screen startups, or do due diligence on YC companies.

日本語の概要は準備中です。原文の説明を表示しています。

Monitor and evaluate RAG systems with retrieval quality metrics, groundedness checks, hallucination detection, and continuous regression testing.

日本語の概要は準備中です。原文の説明を表示しています。

Scaffold a first eval suite for an agent: mine real failures into cases, write behavioural checks over traces, and generate the runner. Use when the user wants evals, regression tests for an agent, a golden set, or asks how to know a prompt/model change did not break things. Do NOT use for a single task's done-check (goal-test) or for auditing context (context-audit).

日本語の概要は準備中です。原文の説明を表示しています。

Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR). Use when the user says "is my dataset okay", "evaluate my data", "check my training data", "I have my own data", or before starting any fine-tuning job. Detects file format, checks schema compliance against the selected model and technique, and reports whether the data is ready for training or evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate and compare technology stacks with TCO analysis, security assessment, and ecosystem health scoring. Use when comparing frameworks, calculating total cost of ownership, assessing migration paths, or analyzing ecosystem viability.

日本語の概要は準備中です。原文の説明を表示しています。

Prompt engineering and LLM evaluation. Use when optimizing prompts, designing prompt templates, evaluating LLM outputs, building agentic systems, implementing RAG, creating few- shot examples, or designing structured-output workflows.

日本語の概要は準備中です。原文の説明を表示しています。

Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.

日本語の概要は準備中です。原文の説明を表示しています。

Score one piece via eval-runner on six dimensions with a fix list. Many pieces → eval-suite. "score this draft before it ships"

日本語の概要は準備中です。原文の説明を表示しています。

Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a custom task, or run an arena-style evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Performance scorecard design, post-matter feedback collection, QBR preparation, and firm comparison for in-house legal ops teams evaluating outside counsel. Design a scorecard framework calibrated to your team's maturity level. Produce post-matter feedback forms for supervising attorneys to complete at matter close. Prepare a QBR pack with agenda, data summary, and talking points for structured business reviews. Produce a side-by-side firm comparison for panel decision-making. Trigger on: 'design a performance scorecard', 'how do we evaluate our firms', 'build us a scorecard', 'post-matter feedback form', 'feedback at matter close', 'collect feedback on the firm', 'QBR', 'quarterly business review', 'prep for the firm review', 'what should I raise with the partner', 'compare the firms', 'which firm is performing better', 'rank our panel', 'firm performance data', 'scorecard framework', 'evaluate outside counsel', 'annual firm review', 'how are our firms performing'.

日本語の概要は準備中です。原文の説明を表示しています。

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

日本語の概要は準備中です。原文の説明を表示しています。

Generate retrieval practice questions at varied difficulty levels for a topic or concept. Use when creating quiz starters, revision activities, or low-stakes testing materials.

日本語の概要は準備中です。原文の説明を表示しています。

Analyze website traffic, global rank, engagement metrics, and traffic sources for any domain. Use this skill to evaluate affiliate program websites, compare competitor traffic, assess advertiser strength, or understand where an audience comes from. Triggers on: "analyze traffic for [domain]", "how much traffic does [site] get", "compare traffic between [site A] and [site B]", "is [program] worth promoting based on traffic", "traffic analysis", "website analytics for [domain]", "where does [site] get traffic", "check if [advertiser] is legit", "evaluate [program] website health", "SimilarWeb analysis", "traffic sources for [domain]", "how popular is [site]", "website rank", "domain authority check", "compare affiliate program websites".

日本語の概要は準備中です。原文の説明を表示しています。

Use when a leader evaluates a marketing TEAM MEMBER using data — a KPI scorecard by role, strengths with real examples, areas to improve stated as observable behavior, a 30 and 90 day development plan, and a 1-5 rating rubric. Trigger on 'performance review for my marketer', 'evaluate a team member', 'monthly 1:1 review', 'feedback for my content writer', 'development plan for an employee', 'is this person underperforming'. Also use when the user is unhappy with someone and cannot articulate why. Not for — evaluating a campaign rather than a person, see `63-campaign-retrospective-global`; reviewing one piece of output, see `62-marketing-review-global`; scoring an external vendor, see `67-agency-vendor-brief-global`.

日本語の概要は準備中です。原文の説明を表示しています。

Guide plugin development workflow — editing skills, agents, hooks, or eval framework in this repo. Use when modifying files in plugins/elixir-phoenix/, lab/eval/, or lab/autoresearch/. Ensures changes pass eval, lint, and tests before committing.

日本語の概要は準備中です。原文の説明を表示しています。

Pattern for progressively refining context retrieval to solve the subagent context problem

日本語の概要は準備中です。原文の説明を表示しています。