「lua」の検索結果

Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate an AI workflow against representative cases and its permitted actions. Use for model, retrieval or agent evaluation; tests do not grant release authority.

日本語の概要は準備中です。原文の説明を表示しています。

Validates dataset formatting and quality for SageMaker model fine-tuning (SFT, DPO, or RLVR). Use when the user says "is my dataset okay", "evaluate my data", "check my training data", "I have my own data", or before starting any fine-tuning job. Detects file format, checks schema compliance against the selected model and technique, and reports whether the data is ready for training or evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate and compare technology stacks with TCO analysis, security assessment, and ecosystem health scoring. Use when comparing frameworks, calculating total cost of ownership, assessing migration paths, or analyzing ecosystem viability.

日本語の概要は準備中です。原文の説明を表示しています。

Prompt engineering and LLM evaluation. Use when optimizing prompts, designing prompt templates, evaluating LLM outputs, building agentic systems, implementing RAG, creating few- shot examples, or designing structured-output workflows.

日本語の概要は準備中です。原文の説明を表示しています。

This skill should be used when the user asks to "evaluate LLM output quality", "set up LLM-as-judge", "build an eval rubric", "compare model outputs pairwise", or "measure agent quality".

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user has nothing — no traces, no labels, no eval set — and needs to build a v0 evaluation from scratch. Also use when the user says "I need to start evaluating my app but don't know where to begin," "I want to set up eval for a new product," or has just identified failure modes and needs to turn them into principles. Outputs a v0 grader in 30 minutes using OpenJudge SimpleRubricsGenerator, plus a roadmap to reach calibrated evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.

日本語の概要は準備中です。原文の説明を表示しています。

Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a custom task, or run an arena-style evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Performance scorecard design, post-matter feedback collection, QBR preparation, and firm comparison for in-house legal ops teams evaluating outside counsel. Design a scorecard framework calibrated to your team's maturity level. Produce post-matter feedback forms for supervising attorneys to complete at matter close. Prepare a QBR pack with agenda, data summary, and talking points for structured business reviews. Produce a side-by-side firm comparison for panel decision-making. Trigger on: 'design a performance scorecard', 'how do we evaluate our firms', 'build us a scorecard', 'post-matter feedback form', 'feedback at matter close', 'collect feedback on the firm', 'QBR', 'quarterly business review', 'prep for the firm review', 'what should I raise with the partner', 'compare the firms', 'which firm is performing better', 'rank our panel', 'firm performance data', 'scorecard framework', 'evaluate outside counsel', 'annual firm review', 'how are our firms performing'.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

日本語の概要は準備中です。原文の説明を表示しています。

Analyze website traffic, global rank, engagement metrics, and traffic sources for any domain. Use this skill to evaluate affiliate program websites, compare competitor traffic, assess advertiser strength, or understand where an audience comes from. Triggers on: "analyze traffic for [domain]", "how much traffic does [site] get", "compare traffic between [site A] and [site B]", "is [program] worth promoting based on traffic", "traffic analysis", "website analytics for [domain]", "where does [site] get traffic", "check if [advertiser] is legit", "evaluate [program] website health", "SimilarWeb analysis", "traffic sources for [domain]", "how popular is [site]", "website rank", "domain authority check", "compare affiliate program websites".

日本語の概要は準備中です。原文の説明を表示しています。

Use when a leader evaluates a marketing TEAM MEMBER using data — a KPI scorecard by role, strengths with real examples, areas to improve stated as observable behavior, a 30 and 90 day development plan, and a 1-5 rating rubric. Trigger on 'performance review for my marketer', 'evaluate a team member', 'monthly 1:1 review', 'feedback for my content writer', 'development plan for an employee', 'is this person underperforming'. Also use when the user is unhappy with someone and cannot articulate why. Not for — evaluating a campaign rather than a person, see `63-campaign-retrospective-global`; reviewing one piece of output, see `62-marketing-review-global`; scoring an external vendor, see `67-agency-vendor-brief-global`.

日本語の概要は準備中です。原文の説明を表示しています。

Post-run self-evaluation system that scores agent output on correctness, clarity, actionability, and conciseness. Use after /team runs, skill executions, or when explicitly asked to evaluate output quality.

日本語の概要は準備中です。原文の説明を表示しています。

Implement evaluated findings: load code-style rules, apply the accepted fixes, then close out the change with full or quick QA. Use after $evaluate-findings has tagged findings and they need to be implemented, or when the user asks to "resolve findings", "apply evaluated findings", or "dispatch findings to implementation".

日本語の概要は準備中です。原文の説明を表示しています。

Critically assess external feedback (code reviews, AI reviewers, PR comments) and decide which suggestions to apply using adversarial verification. Use when the user asks to "evaluate findings", "assess review comments", "triage review feedback", "evaluate review output", or "filter false positives".

日本語の概要は準備中です。原文の説明を表示しています。

Implement evaluated findings: load code-style rules, apply the accepted fixes, then close out the change with full or quick QA. Use after /evaluate-findings has tagged findings and they need to be implemented, or when the user asks to "resolve findings", "apply evaluated findings", or "dispatch findings to implementation".

日本語の概要は準備中です。原文の説明を表示しています。

Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

日本語の概要は準備中です。原文の説明を表示しています。

AWS Bedrock AgentCore comprehensive expert for deploying and managing AI agents at scale. Use when working with any AgentCore service including Gateway, Runtime, Memory, Identity, Code Interpreter, Browser, Observability, Agent Registry, or Evaluations. Covers agent deployment, MCP tool integration, credential management, agent discovery, governance workflows, and automated quality assessment. Essential when user mentions AgentCore, agent runtime, agent registry, agent evaluation, MCP gateway, deploy agent, register MCP server, discover agents, evaluate agent quality, agent credentials, or wants to build, deploy, catalog, or monitor AI agents on AWS.

日本語の概要は準備中です。原文の説明を表示しています。

MLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement.

日本語の概要は準備中です。原文の説明を表示しています。

Provides weighted scoring, rubrics, and decision-threshold patterns. Use when designing quality gates, evaluation systems, or decision frameworks.

日本語の概要は準備中です。原文の説明を表示しています。

Use Hugging Face Evaluate to load metrics, comparisons, and measurements; compute and combine results; run evaluator pipelines; create custom modules; troubleshoot optional dependencies, cache, Hub, and CLI workflows.

日本語の概要は準備中です。原文の説明を表示しています。

Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.

日本語の概要は準備中です。原文の説明を表示しています。

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

日本語の概要は準備中です。原文の説明を表示しています。