Skip to content
cccskills
FreePublished on GitHub

agent-evaluation

Evaluate LLM agents and tool-using workflows—task success, tool accuracy, latency/cost, safety, and regression suites. Use when shipping agent features, comparing prompts/models, or debugging agent failures.

The English summary is being prepared. Showing the original description.

See how to install

Files included (1)

  • SKILL.md1.7 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Agent evaluation

What to measure

DimensionExamples
Task successEnd state matches spec (binary or rubric)
Tool useCorrect tool, valid args, no spurious calls
SafetyNo policy violations, no secret leakage
EfficiencyTokens, latency, tool call count
StabilitySame input -> consistent outcome across runs

Workflow

  1. Define tasks — realistic user intents with clear pass/fail or scored rubric.
  2. Build dataset — golden set + edge cases (errors, ambiguous input, empty context).
  3. Run baseline — fixed model/settings; log traces (inputs, tools, outputs).
  4. Score — automated checks first; human review for ambiguous cases.
  5. Compare — A/B prompts, models, or tool schemas; report deltas with confidence notes.
  6. Gate — block release on regression in must-pass tasks.

Automated checks

  • Schema validation on tool arguments.
  • Assert final answer contains required fields or avoids forbidden content.
  • Snapshot tests for deterministic sub-steps where possible.

Human rubric (when needed)

Score 1-5 on: correctness, completeness, tone, safety. Document disagreements.

Anti-patterns

  • Eval only on cherry-picked happy paths.
  • Changing task and model simultaneously without isolation.
  • No trace logs when debugging tool failures.

Output

Summary table: variant | success rate | avg tools | avg latency | notes.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

"Implement and select ad bidding strategies from manual CPC to automated target-CPA and target-ROAS. Use this skill when the user needs to choose a bidding strategy, set up automated bidding, or optimize bid parameters — even if they say 'what bidding strategy should I use', 'target CPA setup', or 'smart bidding configuration'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

"Optimize advertising budget allocation across campaigns using marginal returns analysis. Use this skill when the user needs to distribute budget across multiple campaigns, optimize spend pacing, or maximize overall ROAS under budget constraints — even if they say 'how to split my ad budget', 'campaign budget optimization', or 'diminishing returns on ad spend'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

"Build CTR prediction models for estimating ad click-through rates from features. Use this skill when the user needs to predict click probability, build an ad ranking model, or evaluate ad creative performance — even if they say 'predict click rate', 'ad relevance scoring', or 'which ad will get more clicks'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

"Implement Generalized Second Price auction for ad slot allocation and pricing. Use this skill when the user needs to understand search ad auctions, compute ad positions and costs-per-click, or analyze bidding dynamics — even if they say 'how does Google Ads auction work', 'ad rank calculation', or 'second price auction for ads'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

"Implement VCG mechanism for incentive-compatible ad slot allocation with truthful bidding. Use this skill when the user needs to design a truthful auction mechanism, compute externality-based payments, or understand why platforms may prefer GSP over VCG — even if they say 'truthful auction design', 'VCG payments', or 'incentive-compatible mechanism'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

"Explain blockchain fundamentals including distributed ledger architecture, consensus mechanisms, and block structure. Use this skill when the user needs to understand blockchain concepts, evaluate whether blockchain fits a use case, or design a blockchain-based solution — even if they say 'how does blockchain work', 'do I need blockchain', or 'distributed ledger'.".

The English summary is being prepared. Showing the original description.

charlieviettq/awesome-agent-skill26Updated Jul 20, 2026

See all skills by charlieviettq

Report a problem with this skill