Skip to content
cccskills
FreePublished on GitHub

regression-watch

Detect quality and efficiency regressions over time using Agent Monitor data — rising error rate (APIError events), falling cache hit rate, growing compaction frequency, and climbing cost-per-session. Splits history into an earlier baseline window and a recent window and reports which metrics are getting worse, by how much, and where. Use when checking whether things are degrading or trending in the wrong direction.

The English summary is being prepared. Showing the original description.

Run score Not tested

See how to install

Files included (2)

  • SKILL.md4.0 KB
  • agents/openai.yaml270 B

SKILL.md (original)

Review the instructions the agent will receive before you install.

Regression Watch

Detect whether Claude Code sessions are getting worse over time across quality and efficiency metrics, using Agent Monitor data.

Input

The user provides: $ARGUMENTS

This may be:

  • empty or "all" — check every regression metric (default)
  • "errors" — error-rate regression only
  • "cache" — cache hit-rate regression only
  • "compaction" — compaction-frequency regression only
  • "cost" — cost-per-session regression only
  • A window like "last 30d" or "30 vs 90" — set the recent vs baseline window sizes

Data Sources

EndpointReturns
GET /api/analyticsdaily_events (365d), daily_sessions (365d), event_types, tokens (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), avg_events_per_session
GET /api/events?session_id=XEvent stream incl. APIError, Compaction, PreToolUse/PostToolUse — used to localize regressions to specific sessions
GET /api/pricing/cost{ total_cost, breakdown[...] } — total cost to derive cost-per-session
GET /api/pricing/cost/{sessionId}Per-session cost — used to compare recent vs baseline session cost
GET /api/workflows/{sessionId}compaction (impact), errorPropagation (by depth), effectiveness — per-session quality signals
GET /api/sessions?limit=NSessions with started_at, cost, metadata — to bucket sessions into time windows

Report Sections

1. Windowing

Split history into a baseline window (older) and a recent window (newer). Default: recent = last 30 days, baseline = the 30–90 day range before it. Use daily_events/daily_sessions for series metrics and GET /api/sessions?limit=N to assign sessions to each window by started_at.

2. Error Rate Regression

  • Recent error rate = APIError count / total events in the recent window (from event_types and daily_events, or per-session GET /api/events).
  • Compare to the baseline rate. Flag if recent is higher.
  • Report the absolute and relative change and which sessions contributed most APIError events.

3. Cache Hit Rate Regression

  • Cache hit rate = total_cache_read / (total_cache_read + total_input).
  • Compute for each window (per-window input/cache_read from session metadata or the pricing breakdown). Flag a falling hit rate — that means more uncached input tokens and higher cost.

4. Compaction Frequency Regression

  • Compaction frequency = Compaction events / session per window (from event_types / daily_events, confirmed via per-session GET /api/workflows/{id} compaction). Flag a rising rate — context is overflowing more often.

5. Cost-per-Session Regression

  • Cost-per-session = window total cost / window session count, using GET /api/pricing/cost overall and GET /api/pricing/cost/{id} for the sessions in each window. Flag a climbing value.

6. Verdict

Roll up which metrics regressed, rank by relative worsening, and name the most likely driver (e.g., cache hit rate fell → cost per session climbed).

Output

  • A Markdown table: metric | baseline | recent | Δ | direction (▲ worse / ▼ better) | verdict.
  • Tag each regressed metric 🔴 (clear regression), 🟡 (mild/within noise), or 🟢 (improved).
  • Currency in USD to 4 decimals; rates as percentages to 2 decimals.
  • List the specific session IDs that contributed most to any regression.
  • End with the single highest-priority regression to address and a concrete next step.
  • Read-only: only report what the API returns; never fabricate baselines.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Inspect fired CCAM alerts and manage alert rules for token thresholds, event patterns, inactivity, and status duration. Use when acknowledging alerts, creating or editing a rule, checking cooldowns, or connecting alert rules to webhook targets.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Identify anomalous sessions using Agent Monitor data — cost outliers from the pricing engine, token anomalies (cache miss spikes, compaction baseline surges), unusual event type ratios (PreToolUse/PostToolUse gaps, APIError clusters), behavioral deviations from workflow intelligence (complexity score outliers, error propagation anomalies), and sessions with abnormal metadata (extreme turn_count, high thinking_blocks, zero turn_duration).

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Produce a detailed report on APIError events from Agent Monitor data — counts over time, which sessions and models are affected, and the likely root cause (rate limits, overload/529, or context-window pressure) inferred from each event's summary and data payload. Use when API errors spike or when you need to explain why requests are failing.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Benchmark one session (or a small recent set) against the rolling average using Agent Monitor data — cost, total tokens, tool count, and workflow complexity score — and report where each metric lands as a percentile of the population. Tells you whether a session was normal, cheap, or an outlier. Use when judging whether a session was typical or out of band.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Define a spend budget for Claude Code and, optionally, create a cost alert rule that fires when usage crosses the limit, via POST /api/alerts/rules on the Agent Monitor dashboard. Reads current spend from /api/pricing/cost to size the budget sensibly and explains every rule field before writing. Use when setting a spend limit or wiring up a budget guardrail.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

Analyze prompt-cache effectiveness for Claude Code usage from the Agent Monitor dashboard — cache hit rate (total_cache_read / (total_cache_read + total_input)), cache_write vs cache_read reuse, cache-read vs cache-write spend, and the sessions with the poorest reuse. Pulls token totals from /api/analytics, per-session detail from /api/sessions, and dollar splits from /api/pricing/cost. Use when diagnosing cache spend or deciding whether prompt caching is paying off.

The English summary is being prepared. Showing the original description.

hoangsonww/Claude-Code-Agent-Monitor1.1KUpdated Oct 23, 2026

See all skills by hoangsonww

Report a problem with this skill