本文へ移動
cccskills
無料GitHub で公開

data-pipeline

Build a production-ready data API from scratch: scaffold REST endpoints with models and validation, generate integration tests that verify every route, then load test for scalability. Use when you need an API backend, data service, CRUD layer, or microservice with verified correctness and performance.

インストール方法を見る

含まれるファイル(1)

  • SKILL.md6.0 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

You are an autonomous data pipeline setup agent. Do NOT ask the user questions. Execute all three phases sequentially without pausing.

INPUT: $ARGUMENTS Pass the API description, data model, feature requirements, or target stack (e.g., "Express + PostgreSQL user management API").

============================================================ PHASE 1: API SCAFFOLD (/api-scaffold)

Follow the instructions defined in the /api-scaffold skill exactly.

Scaffold the complete API layer:

  • Routes and controllers for each resource with RESTful naming
  • Data models with typed fields, relationships, and database schema/migrations
  • Request validation middleware (required fields, types, constraints)
  • Consistent error handling with proper HTTP status codes and error response format
  • Database connection pool, query layer, and transaction support
  • Environment-based configuration (dev/test/prod)

Commit all scaffolded code. Capture the following for Phase 2:

  • Every endpoint path, HTTP method, and expected request/response shape
  • Model names, fields, and relationships
  • Authentication/authorization requirements (if any)

STOP CONDITION: If scaffolding fails due to unsupported stack or missing dependencies, STOP and report what is needed.

============================================================ PHASE 2: INTEGRATION TESTS (/integration-test)

Follow the instructions defined in the /integration-test skill exactly.

Generate integration tests targeting the actual API surface from Phase 1 — not generic templates:

  • Happy path for every endpoint scaffolded in Phase 1
  • Error responses: 400 (bad input), 401 (unauthorized), 404 (not found), 422 (validation failure)
  • Data persistence round-trips: create a resource, then read it back and verify all fields
  • Relationship integrity: foreign key constraints, cascade deletes, orphan prevention
  • Edge cases: empty payloads, max-length strings, special characters, duplicate unique fields
  • Boundary conditions: pagination limits, bulk operations, concurrent writes

Run the tests. On failure:

  • If the test expectation is correct, fix the API code.
  • If the test expectation is wrong, fix the test.
  • Re-run until all pass.

Commit all tests and any API fixes.

============================================================ PHASE 3: LOAD TEST (/load-test)

Follow the instructions defined in the /load-test skill exactly.

Load test the API endpoints from Phase 1 with realistic payloads from the data models:

  • Ramp-up test: 10 -> 100 -> 500 concurrent users
  • Sustained load: target throughput for 60 seconds at expected production traffic
  • Spike test: sudden 5x traffic burst to test graceful degradation
  • Measure: p50, p95, p99 latency; error rate; throughput (requests/sec)

Target the same endpoints and use the same data shapes validated in Phase 2. Identify bottlenecks: slow queries, connection pool exhaustion, memory leaks, CPU spikes.

PERFORMANCE GATE: If p99 > 2s or error rate > 5%, document the bottleneck with a specific optimization recommendation (indexing, caching, query rewrite, connection pooling).

============================================================ SELF-HEALING VALIDATION (max 3 iterations)

After completing all phases, validate the combined output:

  1. Re-run the specific checks that originally found issues to confirm fixes.
  2. Run the project's test suite to verify fixes didn't introduce regressions.
  3. Run build/compile to confirm no breakage.
  4. If new issues surfaced from fixes, add them to the fix queue.
  5. Repeat the fix-validate cycle up to 3 iterations total.

STOP when:

  • Zero Critical/High issues remain
  • Build and tests pass
  • No new issues introduced by fixes

IF STILL FAILING after 3 iterations:

  • Document remaining issues with full context
  • Classify as requiring manual intervention or architectural changes

============================================================ OUTPUT

Data Pipeline Setup Complete

PhaseSkillStatusDetails
1/api-scaffoldPASS/FAIL{N} endpoints, {N} models created
2/integration-testPASS/FAIL{N} tests, {pass}/{fail} results
3/load-testPASS/FAILp95={N}ms, error rate={N}%, throughput={N} rps

API health: {PRODUCTION READY / NEEDS OPTIMIZATION / BROKEN} Bottlenecks: {list any, or "none identified"}

NEXT STEPS:

  • Fix any performance bottlenecks identified in the load test
  • Run /security-review to audit the API for vulnerabilities
  • Run /full-deploy to containerize and set up CI/CD
  • Add rate limiting and caching if load test showed high latency

============================================================ SELF-EVOLUTION TELEMETRY

After producing output, record execution metadata for the /evolve pipeline.

Check if a project memory directory exists:

  • Look for the project path in ~/.claude/projects/
  • If found, append to skill-telemetry.md in that memory directory

Entry format:

### /data-pipeline — {{YYYY-MM-DD}}
- Outcome: {{SUCCESS | PARTIAL | FAILED}}
- Self-healed: {{yes — what was healed | no}}
- Iterations used: {{N}} / {{N max}}
- Bottleneck: {{phase that struggled or "none"}}
- Suggestion: {{one-line improvement idea for /evolve, or "none"}}

Only log if the memory directory exists. Skip silently if not found. Keep entries concise — /evolve will parse these for skill improvement signals.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

508-audit

無料

Run a full Section 508 + WCAG 2.2 Level AA accessibility audit on a federal / .gov / public-sector website or ICT product. Triggers: "Section 508", "508 audit", "WCAG", "VPAT", "ACR".

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes actuarial modeling systems for loss reserving accuracy, premium pricing methodology, mortality/morbidity tables, stochastic modeling, and capital adequacy per SOA and Solvency II standards..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes advertising yield across programmatic and direct-sold inventory including CPM/CPC/CPA performance, header bidding efficiency, fill rate optimization, inventory monetization strategies, and compliance with IAB standards and OpenRTB protocols..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

adr

無料

Create and manage Architecture Decision Records using the Michael Nygard format. Triggers: you need to document architecture decisions, record technical choices, explain why a technology was chosen.

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Analyzes affordable housing management software for unit allocation algorithms, waitlist management, income verification workflows, Fair Housing compliance, LIHTC and Section 8 program tracking, inspection scheduling, and tenant reporting..

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

Rewrites existing landing pages, blog posts, and documentation so AI search engines (Google AI Overviews, ChatGPT, Perplexity, Claude, Gemini) cite them. Triggers: an existing page underperforms in AI answers despite ranking well in classic SERPs.

日本語の概要は準備中です。原文の説明を表示しています。

tinh2/skills-hub-registry192026年9月5日 更新

tinh2 のスキルをすべて見る

このスキルの問題を報告する