跳到正文
cccskills
免费在 GitHub 公开

regression-alert

Compare this period's reliability against the prior period using Agent Monitor data — error rate (APIError/total) and tool-failure rate (PreToolUse→PostToolUse gap) — flag any regression where reliability got worse, and optionally wire a persistent alert rule so the dashboard catches the next regression automatically. Use when checking whether reliability degraded.

中文简介正在准备中。当前显示原文说明。

实测评分 未实测

查看安装方法

包含的文件(2)

  • SKILL.md3.6 KB
  • agents/openai.yaml271 B

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Regression Alert

Detect whether Claude Code reliability is getting worse period-over-period, and optionally arm an alert so it never has to be checked by hand again. Scope is reliability/failures only — for cache/cost/compaction drift, use ccam-insights' regression-watch instead.

Input

The user provides: $ARGUMENTS

This may be:

  • empty or "all" — check error rate and tool-failure rate (default)
  • "errors" — APIError-rate regression only
  • "tools" — tool-failure-rate regression only
  • a window like "7 vs 7" or "30 vs 30" — recent vs baseline window sizes (default: last 7 days vs the prior 7)
  • "arm" — after reporting, also create an alert rule via POST /api/alerts/rules (only on explicit request)

Data Sources

EndpointReturns
GET /api/analyticsdaily_events (365d), daily_sessions (365d), event_types — split into recent vs baseline windows to compute per-window failure rates
GET /api/events?session_id=XPer-session stream — localize a regression to the sessions driving it
GET /api/alerts/rulesExisting alert rules — check whether a matching reliability rule already exists before arming a new one
POST /api/alerts/rulesCreate a new alert rule (only when the user says "arm")

Report Sections

1. Windowing

Split history into a recent window (newer) and a baseline window (the equal-length period just before it). Default: recent = last 7 days, baseline = the prior 7. Use daily_events/daily_sessions to bucket counts by day.

2. Error-Rate Regression

  • Per window: error rate = APIError count / total events.
  • Compare recent vs baseline. Flag if recent is higher. Report absolute change (pp) and relative change (%), plus the recent sessions contributing the most APIError events.

3. Tool-Failure-Rate Regression

  • Per window: tool-failure rate = (PreToolUse − PostToolUse) / PreToolUse.
  • Compare recent vs baseline. Flag a rising rate as a reliability regression. Name the tools whose gap grew most.

4. Verdict

Roll up which rates regressed, rank by relative worsening, and name the most likely driver.

5. Optional — Arm an Alert

Only if the user passed "arm". First GET /api/alerts/rules to avoid duplicates. Then POST /api/alerts/rules with a rule that fires when the regressed metric crosses a threshold near the recent value (e.g., error rate > recent rate). Echo the created rule back; do not create webhooks or fire alerts.

Output

  • A Markdown table: metric | baseline | recent | Δ (pp) | Δ (%) | direction (▲ worse / ▼ better) | verdict.
  • Tag each metric 🔴 (clear regression), 🟡 (within noise), or 🟢 (improved).
  • Rates as percentages to 2 decimals; any currency in USD to 4 decimals.
  • List the specific session IDs that contributed most to any regression.
  • End with the single highest-priority regression and a concrete next step (and, if armed, the new rule's id/threshold).
  • Read-only except the explicit "arm" path, which is the only write. Never mutate alert rules otherwise. If curl cannot reach http://localhost:4820, tell the user to start the dashboard with npm start from the repo root.

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Inspect fired CCAM alerts and manage alert rules for token thresholds, event patterns, inactivity, and status duration. Use when acknowledging alerts, creating or editing a rule, checking cooldowns, or connecting alert rules to webhook targets.

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

Identify anomalous sessions using Agent Monitor data — cost outliers from the pricing engine, token anomalies (cache miss spikes, compaction baseline surges), unusual event type ratios (PreToolUse/PostToolUse gaps, APIError clusters), behavioral deviations from workflow intelligence (complexity score outliers, error propagation anomalies), and sessions with abnormal metadata (extreme turn_count, high thinking_blocks, zero turn_duration).

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

Produce a detailed report on APIError events from Agent Monitor data — counts over time, which sessions and models are affected, and the likely root cause (rate limits, overload/529, or context-window pressure) inferred from each event's summary and data payload. Use when API errors spike or when you need to explain why requests are failing.

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

benchmark

免费

Benchmark one session (or a small recent set) against the rolling average using Agent Monitor data — cost, total tokens, tool count, and workflow complexity score — and report where each metric lands as a percentile of the population. Tells you whether a session was normal, cheap, or an outlier. Use when judging whether a session was typical or out of band.

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

Define a spend budget for Claude Code and, optionally, create a cost alert rule that fires when usage crosses the limit, via POST /api/alerts/rules on the Agent Monitor dashboard. Reads current spend from /api/pricing/cost to size the budget sensibly and explains every rule field before writing. Use when setting a spend limit or wiring up a budget guardrail.

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

Analyze prompt-cache effectiveness for Claude Code usage from the Agent Monitor dashboard — cache hit rate (total_cache_read / (total_cache_read + total_input)), cache_write vs cache_read reuse, cache-read vs cache-write spend, and the sessions with the poorest reuse. Pulls token totals from /api/analytics, per-session detail from /api/sessions, and dollar splits from /api/pricing/cost. Use when diagnosing cache spend or deciding whether prompt caching is paying off.

中文简介正在准备中。当前显示原文说明。

hoangsonww/Claude-Code-Agent-Monitor11102026年10月23日 更新

查看 hoangsonww 的全部技能

报告此技能的问题