本文へ移動
cccskills
無料GitHub で公開

monitoring

Set up application observability with Prometheus, Grafana, Datadog, or CloudWatch — instrument metrics endpoints, configure Golden Signal dashboards, define alert rules with burn-rate SLOs, and add structured logging

インストール方法を見る

含まれるファイル(1)

  • SKILL.md11.0 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

You are in AUTONOMOUS MODE. Do NOT ask questions. Do NOT pause for confirmation. Execute every phase below in sequence, making decisions based on what you find.

============================================================ PHASE 0 — INPUT

$ARGUMENTS may contain:

  • A monitoring stack: prometheus, datadog, cloudwatch, newrelic, grafana
  • --instrument — add application-level metrics instrumentation to source code
  • --alerts-only — generate alerting rules without full dashboard setup
  • --slo — define and configure SLO/SLI targets with burn-rate alerts
  • A specific focus: latency, errors, traffic, saturation
  • If no arguments, auto-detect existing monitoring and extend it, or default to Prometheus + Grafana

============================================================ PHASE 1 — INFRASTRUCTURE DETECTION

Scan for existing monitoring setup:

Prometheus ecosystem:

  • prometheus.yml, prometheus/ directory
  • alertmanager.yml, alertmanager/ directory
  • Grafana dashboards: grafana/, dashboards/, *.json with "panels" key
  • Docker compose services named prometheus, grafana, alertmanager

Cloud-native:

  • AWS: CloudWatch references in Terraform, aws_cloudwatch_* resources
  • GCP: Cloud Monitoring, google_monitoring_* resources
  • Azure: Application Insights, azurerm_monitor_* resources

Third-party:

  • Datadog: datadog.yaml, DD_* environment variables, datadog-agent in compose
  • New Relic: .newrelic.yml, NEW_RELIC_* env vars, newrelic in dependencies
  • Sentry: sentry.properties, SENTRY_DSN in env, @sentry/* in deps

Application stack:

  • Node.js: check for prom-client, express-prometheus-middleware, @opentelemetry/*
  • Python: check for prometheus_client, django-prometheus, opentelemetry-*
  • Go: check for prometheus/client_golang, go.opentelemetry.io
  • Java: check for Micrometer, Spring Actuator

Infrastructure:

  • Kubernetes: check for ServiceMonitor CRDs, Prometheus Operator
  • Docker: check compose services for metrics endpoints
  • Serverless: check for X-Ray, CloudWatch Logs

============================================================ PHASE 2 — METRICS DESIGN (Golden Signals)

Design metrics based on the Four Golden Signals:

1. Latency — time to service a request

  • http_request_duration_seconds (histogram)
  • Buckets: 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10
  • Track by: method, route, status_code
  • Report P50, P90, P95, P99 percentiles

2. Traffic — demand on the system

  • http_requests_total (counter)
  • Track by: method, route, status_code
  • Report requests per second

3. Errors — rate of failed requests

  • http_errors_total (counter) — 5xx responses
  • http_client_errors_total (counter) — 4xx responses
  • Error rate = errors / total requests

4. Saturation — how full the service is

  • CPU utilization, memory usage
  • Connection pool usage (database, Redis)
  • Queue depth (if applicable)
  • Event loop lag (Node.js)
  • GC pause time (Java, Go)

============================================================ PHASE 3 — APPLICATION INSTRUMENTATION (if --instrument)

Add metrics middleware to the application:

Node.js (Express/Fastify):

  • Install prom-client dependency
  • Create metrics middleware tracking http_request_duration_seconds and http_requests_total
  • Add /metrics endpoint exposing Prometheus format
  • Add default Node.js process metrics (memory, CPU, event loop)

Python (FastAPI/Django):

  • Install prometheus_client or prometheus-fastapi-instrumentator
  • Expose /metrics endpoint
  • Add middleware for request duration tracking

Go:

  • Add promhttp.Handler() on /metrics
  • Use promauto for auto-registering metrics
  • Add middleware using promhttp.InstrumentHandlerDuration

Also add to the application:

  • Health endpoints: /health (liveness), /ready (readiness)
  • Version endpoint: /version returning app version and build info
  • Structured logging: ensure logs are JSON-formatted with correlation IDs

============================================================ PHASE 4 — PROMETHEUS + GRAFANA SETUP

If using Prometheus stack, generate:

monitoring/prometheus/prometheus.yml:

  • Global scrape interval: 15s, evaluation interval: 15s
  • Rule files referencing alerts/*.yml
  • Scrape config targeting app:{port} on /metrics with 10s interval

monitoring/prometheus/alerts/app.yml: Alert rules (all with for duration to prevent flapping):

  • HighErrorRate: rate(http_errors_total[5m]) / rate(http_requests_total[5m]) > 0.01 for 5m (critical)
  • HighLatency: P99 > 2s for 5m (warning)
  • HighMemoryUsage: > 512MB for 10m (warning)
  • HighCPU: > 80% for 10m (warning)
  • DiskUsageHigh: > 80% for 5m (warning)
  • ServiceDown: up == 0 for 1m (critical)

monitoring/grafana/dashboards/app.json: Grafana dashboard JSON with panels:

  1. Request Rate (by status code)
  2. Error Rate (percentage over time)
  3. Latency Distribution (heatmap)
  4. P50/P90/P99 Latency (time series)
  5. Active Connections (gauge)
  6. Memory Usage (time series)
  7. CPU Usage (time series)
  8. Saturation (connection pool, queue depth)

monitoring/grafana/provisioning/dashboards.yml: File-based dashboard provisioning.

Docker Compose (monitoring/docker-compose.monitoring.yml):

  • Prometheus v2.51+ with 30d retention
  • Grafana 10.4+ with provisioned dashboards and datasources
  • Alertmanager v0.27+ for notification routing
  • All with proper volume mounts and health checks

============================================================ PHASE 5 — CLOUD MONITORING (if Datadog/CloudWatch/New Relic)

Datadog:

  • Generate datadog.yaml agent config
  • Add dd-trace to application dependencies
  • Configure APM, log collection, custom metrics
  • Generate monitor definitions in JSON

CloudWatch:

  • Generate Terraform for CloudWatch alarms, dashboards, log groups
  • Configure metric filters on log groups
  • Set up SNS topics for alarm notifications

New Relic:

  • Generate newrelic.yml configuration
  • Add agent to application dependencies
  • Configure custom dashboards via NR API (NRQL queries)

============================================================ PHASE 6 — SLO CONFIGURATION (if --slo)

Define SLOs based on service type:

  • Availability SLO: 99.9% uptime (43.8 min/month error budget)
  • Latency SLO: 95% of requests < 200ms, 99% < 1s
  • Error SLO: < 0.1% error rate

Generate burn rate alerts:

  • Fast burn (2%/hour): page immediately — requires human attention within minutes
  • Slow burn (5%/day): ticket within 1 hour — investigate during business hours

============================================================ SELF-HEALING VALIDATION (max 2 iterations)

After completing deployment/infrastructure changes, validate:

  1. Verify all generated files are syntactically valid (YAML, JSON, HCL, Dockerfile).
  2. Run validation commands if available (terraform validate, docker build --check, kubectl dry-run).
  3. Verify no secrets, credentials, or sensitive values are hardcoded.
  4. If validation fails, diagnose and fix the specific syntax or config error.
  5. Repeat up to 2 iterations.

IF STILL FAILING after 2 iterations:

  • Document what failed and the exact error
  • Include partial output if available

============================================================ OUTPUT

## Monitoring Setup Complete

### Stack: {Prometheus + Grafana / Datadog / CloudWatch / New Relic}

### Files Created
{list of all generated files with one-line descriptions}

### Metrics Endpoints
- Application: http://localhost:{port}/metrics
- Prometheus: http://localhost:9090
- Grafana: http://localhost:3001 (admin/admin)
- Alertmanager: http://localhost:9093

### Alert Rules
| Alert | Condition | Severity |
|-------|-----------|----------|
| HighErrorRate | >1% for 5m | critical |
| HighLatency | P99 >2s for 5m | warning |
| HighMemory | >512MB for 10m | warning |
| DiskUsageHigh | >80% for 5m | warning |
| ServiceDown | down for 1m | critical |

### Dashboard Panels
{list of dashboard panels with their metric queries}

============================================================ NEXT STEPS

  1. Start monitoring stack: docker compose -f monitoring/docker-compose.monitoring.yml up -d
  2. Verify metrics are being scraped: check Prometheus targets page at :9090/targets
  3. Configure alerting notification channels (Slack, PagerDuty, email) in Alertmanager
  4. Add application-specific custom metrics for business KPIs
  5. Set up log aggregation if not already configured (Loki, ELK)
  6. Review alert thresholds after 1 week of baseline data

============================================================ SELF-EVOLUTION TELEMETRY

After producing output, record execution metadata for the /evolve pipeline.

Check if a project memory directory exists:

  • Look for the project path in ~/.claude/projects/
  • If found, append to skill-telemetry.md in that memory directory

Entry format:

### /monitoring — {{YYYY-MM-DD}}
- Outcome: {{SUCCESS | PARTIAL | FAILED}}
- Self-healed: {{yes — what was healed | no}}
- Iterations used: {{N}} / {{N max}}
- Bottleneck: {{phase that struggled or "none"}}
- Suggestion: {{one-line improvement idea for /evolve, or "none"}}

Only log if the memory directory exists. Skip silently if not found. Keep entries concise — /evolve will parse these for skill improvement signals.

============================================================ DO NOT

  • Do NOT set alert thresholds too aggressively — avoid alert fatigue
  • Do NOT expose Prometheus/Grafana ports publicly without authentication
  • Do NOT store Grafana admin passwords in plain text in committed files
  • Do NOT use rate() on gauges — use rate() only on counters and histograms
  • Do NOT create alerts without for duration — always require sustained condition
  • Do NOT use high-cardinality labels (user IDs, request IDs) in Prometheus metrics
  • Do NOT scrape more frequently than every 10s without good reason
  • Do NOT overwrite existing monitoring configs — extend them
  • Do NOT add instrumentation that significantly impacts application performance

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

508-audit

無料

Run a full Section 508 + WCAG 2.2 Level AA accessibility audit on a federal / .gov / public-sector website or ICT product. Triggers: "Section 508", "508 audit", "WCAG", "VPAT", "ACR".

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes actuarial modeling systems for loss reserving accuracy, premium pricing methodology, mortality/morbidity tables, stochastic modeling, and capital adequacy per SOA and Solvency II standards..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes advertising yield across programmatic and direct-sold inventory including CPM/CPC/CPA performance, header bidding efficiency, fill rate optimization, inventory monetization strategies, and compliance with IAB standards and OpenRTB protocols..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

adr

無料

Create and manage Architecture Decision Records using the Michael Nygard format. Triggers: you need to document architecture decisions, record technical choices, explain why a technology was chosen.

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes affordable housing management software for unit allocation algorithms, waitlist management, income verification workflows, Fair Housing compliance, LIHTC and Section 8 program tracking, inspection scheduling, and tenant reporting..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Rewrites existing landing pages, blog posts, and documentation so AI search engines (Google AI Overviews, ChatGPT, Perplexity, Claude, Gemini) cite them. Triggers: an existing page underperforms in AI answers despite ranking well in classic SERPs.

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

tinh2 のスキルをすべて見る

このスキルの問題を報告する