Results for “evaluate”

Evaluate any agent skill against a merged framework — Anthropic's Claude Code best practices plus Matt Pocock's writing-great-skills methodology — across 4 axes (Trigger, Structure, Steering, Pruning). Produces an evidence-cited scorecard (0–100), a weighted overall score, and diagnosed failure modes with prioritized fixes. Use when the user asks to evaluate, rate, or audit a skill ("evaluate this skill", "skill scorecard", "review SKILL.md"), or to compare two skills.

The English summary is being prepared. Showing the original description.

Evaluate an incoming workflow upgrade — a skill, MCP server, subagent, library, external service, prompt, or pattern — through one disciplined pipeline before adopting it. Security is a blocking gate. Records every verdict so the same thing is never evaluated twice. Use when considering a new tool from a post, repo, or recommendation. Triggers — "vet this", "should we adopt", "is this safe to install", "evaluate this skill/MCP", "what do you think of this tool".

The English summary is being prepared. Showing the original description.

Use when running a jevaluate rating round, re-rating after a rubric change, changing the jevaluate rubric, or publishing ratings to the jev-mode ratings folder.

The English summary is being prepared. Showing the original description.

MMI評価エージェント - Modularity Maturity Indexによるモジュール成熟度評価（Cohesion/Coupling/Independence/Reusability）。/evaluate-mmi [対象パス] で呼び出し。

The English summary is being prepared. Showing the original description.

MMI評価エージェント - Modularity Maturity Indexによるモジュール成熟度評価（Cohesion/Coupling/Independence/Reusability）。/evaluate-mmi [対象パス] で呼び出し。

The English summary is being prepared. Showing the original description.

Autonomous model validation and benchmarking. Investigates any ML model (LLM, image gen, TTS, time series, etc.), runs it on GPU cloud, evaluates quality and performance, and generates HTML reports. Use when user asks to verify, benchmark, evaluate, or test a model. Triggers on "verify model", "benchmark", "evaluate model", "test model", "run benchmark", "model evaluation", "モデルを検証", "ベンチマーク", "モデルを試して".

The English summary is being prepared. Showing the original description.

Orchestrator for the pre-decision business-idea lifecycle — generate ideas, model them, evaluate them, validate them with customers, and only then hand off to product-soul / brainstorming. Routes through `idea-generation`, `business-modeling`, `idea-evaluation`, and `customer-discovery`. Load when the user asks to explore business ideas, find a startup idea, evaluate a venture, validate an idea, says "what business should I start", "should I build this", "is this a good business", "I have a startup idea", "evaluate this venture", "model this business", "validate this idea", "Mom Test this", "Lean Canvas this", "Business Model Canvas", "Value Proposition Canvas", "go/no-go on this idea". Pre-decision suite — once one idea is committed, hands off to `product-soul`. Does NOT design features (use `brainstorming`) or audit built products (use `reality-check`).

The English summary is being prepared. Showing the original description.

Score an unbuilt business idea on desirability, viability, feasibility, distribution wedge, why-now, founder-market-fit, market size, alternatives, defensibility, capital intensity, and regulatory/ethical risk — and return a GO / ITERATE / KILL verdict with kill criteria and a next kill test. Load when the user asks to evaluate a business idea, score a startup idea, screen an idea, decide whether to pursue this venture, do an idea review, or says "is this a good business idea", "should I build this", "evaluate this startup", "screen this idea", "go/no-go on this idea", "kill or pursue". Sub-skill of `venture-exploration`. Calls `fermi` for sizing, `assumption-mapping` for hidden beliefs, optional `pre-mortem` / `adversarial-hat` for high-stakes ideas. Does NOT evaluate built products — for that use `reality-check`.

The English summary is being prepared. Showing the original description.

Evaluate any project, product, or system's claims against its actual implementation — scoring each claim for truth, identifying architectural gaps, assessing competitive positioning, proposing creative solutions, and producing an actionable roadmap. Load when the user asks to evaluate claims, reality-check a project, assess what this project actually does vs what it says, validate product claims, or score a system's credibility. Also triggers on "is this real", "does this work as claimed", "evaluate this project", "assess the gap between claims and reality", "how credible is this", "investor assessment", "score these claims", or "what's real vs marketing".

The English summary is being prepared. Showing the original description.

Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.

The English summary is being prepared. Showing the original description.

Design and refine an AI agent’s available actions, tool inputs, and result formats. Define recovery steps for errors, manage the information kept in context, and evaluate changes through completion rates, retries, and cost per successful task.

• Define an AI agent’s tool set and input formats
• Make tool results and next steps easier to interpret
• Specify safe retries and stopping conditions

Evaluate Caveman optimization observations by selecting a specific code change with operator approval. Compare the original and changed versions on identical inputs, checking both task quality and resource usage before deciding whether to keep the change.

• Inspecting a Caveman observation report
• Comparing token or byte usage before and after a change
• Checking whether an optimization preserves quality

Evaluate substantial software changes against the actual problem, current system, and concrete users or components that need them. Choose a sufficient scope, assess consequences, and, when implementation is requested, make the change and verify its behavior.

• Assessing whether a feature proposal is needed
• Reducing the scope of a refactor or migration
• Checking who needs a proposed API or data field

Learn to create, discover, invoke, secure, evaluate, package, and port Agent Skills through one interactive lesson at a time. Practical checkpoints and quizzes track demonstrated progress in a local file, so later sessions resume where you stopped.

• Learn to build and invoke a minimal Agent Skill
• Understand discovery and automatic routing
• Check permission, sandbox, and trust boundaries

Evaluate an Agent Skill bundle before release by checking structure, activation conditions, output quality, scripts, safety, file integrity, and host compatibility. Receive a detailed JSON report that distinguishes local evidence checks from production readiness backed by external attestation.

• Reviewing a skill bundle before distribution
• Evaluating activation accuracy and output improvements
• Checking scripts, safety, and distributed files

Plan and evaluate free calculators, generators, or assessment tools for marketing. Compare ideas against audience needs, search demand, development effort, and maintenance, then define an initial version and a path toward your paid product.

• Comparing free tool ideas for attracting prospects
• Defining the first version of an ROI calculator
• Deciding when to request an email address

Design and evaluate how an AI agent remembers user preferences and history across conversations. Compare storage formats and memory frameworks, plan updates and compression, and distinguish personal memory from shared document retrieval while addressing sensitive information.

• Design user memory that persists across conversations
• Compare storage formats, Mem0, and Memobase
• Evaluate recall and cross-conversation reasoning

Evaluate BMad skills using your agent's configured runner. Compare results with the model alone or another version, grade outputs against criteria, and test whether descriptions trigger correctly, preserving transcripts and evidence for each run.

• Compare a skill with the model alone
• Compare an edited skill with a previous version
• Grade outputs against defined criteria

Forecast sales and other time series with Google’s TimesFM without task-specific training. Prepare regularly spaced CSV data, export predicted values and uncertainty ranges, add explanatory variables, and evaluate forecasts against held-out observations.

• Forecast monthly sales or revenue from CSV data
• Export predicted values with uncertainty ranges
• Forecast with additional explanatory variables

Use PyTDC to discover therapeutic machine-learning datasets, create task-appropriate training and test splits, and evaluate predictions against curated benchmarks. Also score selected molecules with supported scoring functions, with explicit download plans and bounded outputs.

• Find datasets for therapeutic machine-learning tasks
• Plan and create task-appropriate data splits
• Evaluate predictions with benchmark-specific metrics

Evaluate job offers by separating guaranteed pay, conditional bonuses, and estimated equity value. Compare terms, develop an evidence-based counteroffer, draft emails and call scripts, rehearse recruiter conversations, and prepare deadline extensions, acceptance, or decline messages.

• Comparing pay and terms across job offers
• Drafting a counteroffer email
• Rehearsing a salary negotiation call

Design backtests that evaluate trading strategies against historical data. Account for trading costs, avoid using future information or excluding delisted securities, and compare strategies using separate development and evaluation periods.

• Build a backtesting system for trading strategies
• Check for bias in historical performance estimates
• Compare strategies after accounting for trading costs

Evaluates currency trades that aim to earn interest rate differences. It combines spot and forward prices, expected volatility, and historical trends to compare potential returns against risk across holding periods and present trade recommendations.

• Comparing currency pairs for interest rate carry
• Choosing a trade tenor using forward curves
• Comparing carry returns against expected volatility

Build an end-to-end evaluation pipeline for Python AI applications using pixie-qa and real language-model calls. Define criteria, prepare realistic test data, score application runs, and analyze the evidence to produce a prioritized improvement plan.

• Setting up quality checks for Python AI applications
• Creating realistic scenarios for known failure modes
• Scoring full application runs with real language-model calls