Skip to content
cccskills
FreePublished on GitHub

typescript-data-engineering

Use when building data pipelines, ETL jobs, event processors, message-broker producers/consumers, application caching layers, database migrations, BigQuery queries, or event-sourcing handlers in TypeScript. Triggers on edits to indexer/ETL/migration code, Prisma or Drizzle schemas, BigQuery integration code, RabbitMQ/Kafka/SQS/BullMQ producer or consumer code, Redis cache wrappers, or mentions of "data engineering", "ETL", "pipeline", "indexer", "event sourcing", "data warehouse", "data migration", "message queue", "RabbitMQ", "Kafka", "SQS", "BullMQ", "Redis cache", or "caching".

The English summary is being prepared. Showing the original description.

See how to install

Files included (11)

  • SKILL.md4.3 KB
  • references/architecture.md1.1 KB
  • references/bigquery.md1.9 KB
  • references/caching.md7.7 KB
  • references/data-models.md916 B
  • references/etl-pipelines.md1.9 KB
  • references/event-sourcing.md2.3 KB
  • references/message-brokers.md9.5 KB
  • references/migrations-and-infra.md2.9 KB
  • references/orms.md1.5 KB
  • references/validation-and-cron.md1.6 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Data Engineering (TypeScript)

You are operating as a data engineer. Optimize for correctness and replayability over cleverness: every pipeline step must be idempotent, every projection derivable from the immutable event log.

Reference stack: PostgreSQL 17 (Prisma and/or Drizzle), Redis 7, Google BigQuery as the analytics warehouse, and an event-sourcing pipeline that ingests external/blockchain events through an inbox/outbox pattern with exactly-once semantics.

Services may share a database through a generated client package but stay decoupled through events. Scheduled cron jobs handle ETL and projection generation; bulk artifacts (e.g. merkle trees) publish to object storage.

Universal Rules

  1. Idempotency everywhere — every pipeline step must be safe to re-run.
  2. Single source of truth — the ingest event log is immutable; downstream tables are projections of it.
  3. Partition by time — both PostgreSQL indexes and BigQuery tables should partition on timestamps.
  4. Fail loudly — invalid data goes to DLQ, not silently dropped.
  5. Exactly-once semantics — use outbox + deduplication, not "at-most-once" or "hope for the best".
  6. Denormalize for analytics — flatten at ETL time for BigQuery; normalize for PostgreSQL.
  7. Backfill-ready — every projection must support replay from the event log.
  8. Schema evolution — add fields as nullable, never remove or rename in-place.
  9. Validate at boundaries with Zod — not between internal modules.
  10. Outbox in same transaction — every event write must also write its outbox row atomically.

References

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Use when adversarially reviewing a document that makes formal or technical claims — math derivations, physics papers, statistical analyses, benchmark reports, whitepapers. Inventories every equation and quantitative claim, verifies each AS NAMED in the text (never a paraphrase or a neighboring statement), and classifies VERIFIED / REFUTED / UNVERIFIABLE / VACUOUS. Triggers on "check this paper", "verify these claims", "is this derivation right", "review this proof", "audit this benchmark", "does the math hold up". For source-code review see code-review-and-quality; for skill/agent library audits see skill-library-review; for content quality scoring see content-ops.

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

Run Karpathy-style autoresearch optimization on any content. Generates 50+ variants, scores with a 5-expert simulated panel, evolves winners through multiple rounds, outputs optimized version + full experiment log. Use when optimizing landing pages, email sequences, ad copy, headlines, form pages, CTA text, or any conversion-focused content. Triggers on "optimize this page", "run autoresearch", "score these variants", "A/B test this copy".

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

Tests in real browsers. Use when building or debugging anything that runs in a browser. Use when you need to inspect the DOM, capture console errors, analyze network requests, profile performance, or verify visual output with real runtime data via Chrome DevTools MCP.

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

Conducts multi-axis code review across correctness, readability, architecture, security, and performance. Use before merging any change. Use when reviewing code written by yourself, another agent, or a human. Triggers on "review my PR", "review this diff", "code review", "review this changeset", "is this ready to merge", "pre-merge review".

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

Estimate the full development cost of an existing codebase from lines of code, architectural complexity, and team-composition overhead. Use for 'how much would this cost to build', 'what did this codebase cost', development-cost or build-cost estimates, calendar-time estimates, and Claude/AI ROI on a delivered codebase. Estimates by measured LOC and complexity, not by ticket volume.

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

Auto-assembles a domain-specific expert panel (7–10 experts), scores any content or strategy artifact against a typed rubric, and iterates until the aggregate hits 90+ (max 3 rounds). Use as a quality gate on copy, email sequences, landing-page drafts, strategy docs, charts, titles, or recruiting evaluations — or when another skill needs a final review gate on its output. Triggers on "expert panel this", "score this", "rate these variants", "quality check this", "panel review", "expert score", "evaluate this copy/strategy/page". For variant generation and multi-round conversion optimization see autoresearch; for live-URL CRO auditing see conversion-ops; for the scripted content-production pipeline see content-pipeline.

The English summary is being prepared. Showing the original description.

LazyIsEfficient/agentic-os17Updated Jul 17, 2026

See all skills by LazyIsEfficient

Report a problem with this skill