Skip to content
cccskills
FreePublished on GitHub

regression-alert

Compare this period's reliability against the prior period using Agent Monitor data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse gap) — flag any regression where reliability got worse, and optionally wire a persistent alert rule so the dashboard catches the next regression automatically. Use when checking whether reliability degraded.

The English summary is being prepared. Showing the original description.

Run score Not tested

See how to install

Files included (2)

  • SKILL.md3.6 KB
  • agents/openai.yaml271 B

SKILL.md (original)

Review the instructions the agent will receive before you install.

Regression Alert

Detect whether Claude Code reliability is getting worse period-over-period, and optionally arm an alert so it never has to be checked by hand again. Scope is reliability/failures only — for cache/cost/compaction drift, use ccam-insights' regression-watch instead.

Input

The user provides: $ARGUMENTS

This may be:

  • empty or "all" — check error rate and tool-failure rate (default)
  • "errors" — APIError-rate regression only
  • "tools" — tool-failure-rate regression only
  • a window like "7 vs 7" or "30 vs 30" — recent vs baseline window sizes (default: last 7 days vs the prior 7)
  • "arm" — after reporting, also create an alert rule via POST /api/alerts/rules (only on explicit request)

Data Sources

EndpointReturns
GET /api/analyticsdaily_events (365d), daily_sessions (365d), event_types — split into recent vs baseline windows to compute per-window failure rates
GET /api/events?session_id=XPer-session stream — localize a regression to the sessions driving it
GET /api/alerts/rulesExisting alert rules — check whether a matching reliability rule already exists before arming a new one
POST /api/alerts/rulesCreate a new alert rule (only when the user says "arm")

Report Sections

1. Windowing

Split history into a recent window (newer) and a baseline window (the equal-length period just before it). Default: recent = last 7 days, baseline = the prior 7. Use daily_events/daily_sessions to bucket counts by day.

2. Error-Rate Regression

  • Per window: error rate = APIError count / total events.
  • Compare recent vs baseline. Flag if recent is higher. Report absolute change (pp) and relative change (%), plus the recent sessions contributing the most APIError events.

3. Tool-Failure-Rate Regression

  • Per window: tool-failure rate = (PreToolUse − PostToolUse) / PreToolUse.
  • Compare recent vs baseline. Flag a rising rate as a reliability regression. Name the tools whose gap grew most.

4. Verdict

Roll up which rates regressed, rank by relative worsening, and name the most likely driver.

5. Optional — Arm an Alert

Only if the user passed "arm". First GET /api/alerts/rules to avoid duplicates. Then POST /api/alerts/rules with a rule that fires when the regressed metric crosses a threshold near the recent value (e.g., error rate > recent rate). Echo the created rule back; do not create webhooks or fire alerts.

Output

  • A Markdown table: metric | baseline | recent | Δ (pp) | Δ (%) | direction (▲ worse / ▼ better) | verdict.
  • Tag each metric 🔴 (clear regression), 🟡 (within noise), or 🟢 (improved).
  • Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
  • List the specific session IDs that contributed most to any regression.
  • End with the single highest-priority regression and a concrete next step (and, if armed, the new rule's id/threshold).
  • Read-only except the explicit "arm" path, which is the only write. Never mutate alert rules otherwise. If curl cannot reach http://localhost:4820, tell the user to start the dashboard with npm start from the repo root.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Inspect fired CCAM alerts and manage alert rules for token thresholds, event patterns, inactivity, and status duration. Use when acknowledging alerts, creating or editing a rule, checking cooldowns, or connecting alert rules to webhook targets.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Identify anomalous sessions using Agent Monitor data — cost outliers from the pricing engine, token anomalies (cache miss spikes, compaction baseline surges), unusual event type ratios (PreToolUse/PostToolUse gaps, APIError clusters), behavioral deviations from workflow intelligence (complexity score outliers, error propagation anomalies), and sessions with abnormal metadata (extreme turn_count, high thinking_blocks, zero turn_duration).

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Produce a detailed report on APIError events from Agent Monitor data — counts over time, which sessions and models are affected, and the likely root cause (rate limits, overload/529, or context-window pressure) inferred from each event's summary and data payload. Use when API errors spike or when you need to explain why requests are failing.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Benchmark one session (or a small recent set) against the rolling average using Agent Monitor data — cost, total tokens, tool count, and workflow complexity score — and report where each metric lands as a percentile of the population. Tells you whether a session was normal, cheap, or an outlier. Use when judging whether a session was typical or out of band.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Define a spend budget for Claude Code and, optionally, create a cost alert rule that fires when usage crosses the limit, via POST /api/alerts/rules on the Agent Monitor dashboard. Reads current spend from /api/pricing/cost to size the budget sensibly and explains every rule field before writing. Use when setting a spend limit or wiring up a budget guardrail.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Analyze prompt-cache effectiveness for Claude Code usage from the Agent Monitor dashboard — cache hit rate (total_cache_read / (total_cache_read + total_input)), cache_write vs cache_read reuse, cache-read vs cache-write spend, and the sessions with the poorest reuse. Pulls token totals from /api/analytics, per-session detail from /api/sessions, and dollar splits from /api/pricing/cost. Use when diagnosing cache spend or deciding whether prompt caching is paying off.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

See all skills by hoangsonww

Report a problem with this skill