Skip to content
cccskills
FreePublished on GitHub

benchmark-optimization-loop

Turn speedup requests into a measured optimization process with limits on time, cost, and variants. Compare each change against a baseline and correctness checks, then retain the fastest safe option with reproducible commands and measurements.

OriginalShow the original description

Convert 'make it faster' requests into a bounded measured optimization loop — baseline first, generate one-hypothesis variants, benchmark each against a correctness gate, and promote the fastest safe variant with reproducible commands. Use when asked to speed something up, try many variants, run recursive optimization, benchmark latency/throughput/cost, or pick the best implementation by repeated measured tests.

See how to install

Good for

  • Speeding up a slow operation using measured comparisons
  • Comparing implementation variants under the same input conditions
  • Evaluating latency, throughput, or cost per run
  • Setting limits for repeated optimization searches

Files included (1)

  • SKILL.md2.6 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Benchmark Optimization Loop

Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.

Required Baseline

Do not optimize until these exist:

  • the operation being optimized;
  • the correctness gate that must stay green;
  • the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
  • the current baseline;
  • the search budget: max variants, max time, max spend, max data impact.

If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.

Loop

  1. Measure the baseline.
  2. Identify bottlenecks from evidence.
  3. Generate variants that test one hypothesis each.
  4. Run variants with the same input shape.
  5. Reject variants that fail correctness, safety, or reproducibility.
  6. Promote the fastest safe variant.
  7. Codify the winning path in a script, command, test, config, or doc.
  8. Rerun the baseline and winner to confirm the delta.

Variant Table

Track variants like this:

Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited

Recursive Search

For recursive or hyperparameter work:

  • persist every run to a ledger;
  • compare against the prior accepted winner, not only the previous run;
  • keep a holdout or replay check;
  • stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.

Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.

Promotion Gate

A variant cannot become the new default until:

  • correctness tests pass;
  • the performance delta is repeated or explained;
  • rollback is obvious;
  • the change is encoded in source control or a durable runbook;
  • the final summary includes exact commands and measurements.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

accessibility

FreeSummary

Design and audit interfaces for Web, iOS, and Android using WCAG 2.2 accessibility guidance. Check screen-reader labels, keyboard navigation, contrast, and control sizes to identify barriers and guide implementation for people using assistive technology.

  • Adding descriptive labels to icon-only buttons
  • Reviewing keyboard navigation and modal focus
  • Checking contrast and control sizes
affaan-m/ECC283.4KUpdated Oct 16, 2026

Investigate unreliable AI agents across 12 layers, including instructions, memory, tool execution, and display. Use code and logs to trace failures, rank findings by severity, and produce an ordered repair plan that prioritizes code changes.

  • Investigating failures that appear only inside an agent application
  • Tracing old conversation content leaking into new sessions
  • Checking skipped or falsely claimed tool execution
affaan-m/ECC283.4KUpdated Oct 16, 2026

agent-eval

FreeSummaryAI rating 80

Compare coding agents on tasks from your own codebase. Define success checks, run agents in separate Git working directories, and report pass rates, available API costs, completion time, and consistency across repeated trials.

  • Comparing coding agents on real development tasks
  • Evaluating a new tool or model before adoption
  • Checking performance after agent updates
affaan-m/ECC283.4KUpdated Oct 16, 2026

Design and refine an AI agent’s available actions, tool inputs, and result formats. Define recovery steps for errors, manage the information kept in context, and evaluate changes through completion rates, retries, and cost per successful task.

  • Define an AI agent’s tool set and input formats
  • Make tool results and next steps easier to interpret
  • Specify safe retries and stopping conditions
affaan-m/ECC283.4KUpdated Oct 16, 2026

Diagnose AI agent runs that fail repeatedly, loop through tools, or lose track of the task. Capture errors and environment details, test a focused recovery action, and produce a report explaining the evidence and outcome.

  • Diagnosing repeated failures in an agent run
  • Investigating loops through the same tool calls
  • Reviewing output degradation as context grows
affaan-m/ECC283.4KUpdated Oct 5, 2026

agent-introspection-debugging

FreeSummaryAI rating 100

Diagnose failed AI agent runs by capturing errors, checking likely causes, and choosing small, reversible recovery actions. The workflow produces a structured report of the failure, recovery steps, and evidence, helping explain repeated retries or stalled progress.

  • Diagnosing repeated tool calls with no progress
  • Investigating quality loss as context grows
  • Checking file paths and environment mismatches
affaan-m/ECC283.4KUpdated Oct 16, 2026

See all skills by affaan-m

Report a problem with this skill