626 件 ・ 関連度順
概要と使いどころ
Parses complex problems into candidate DAG (Directed Acyclic Graph) execution structures. Decomposes tasks into nodes with dependencies and identifies candidate parallelization opportunities. Activate on 'build dag', 'create workflow graph', 'decompose task', 'execution graph', 'task graph'. NOT for simple linear tasks or when an existing DAG structure is provided.
日本語の概要は準備中です。原文の説明を表示しています。
Profiles DAG execution performance including latency, token usage, cost, and resource consumption. Identifies bottlenecks and optimization opportunities. Activate on 'performance profile', 'execution metrics', 'latency analysis', 'token usage', 'cost analysis'. NOT for execution tracing (use dag-execution-tracer) or failure analysis (use dag-failure-analyzer).
日本語の概要は準備中です。原文の説明を表示しています。
Combines and synthesizes outputs from parallel DAG branches. Handles merge strategies, conflict resolution, and result formatting. Activate on 'aggregate results', 'combine outputs', 'merge branches', 'synthesize results', 'fan-in'. NOT for execution (use dag-parallel-executor) or scheduling (use dag-task-scheduler).
日本語の概要は準備中です。原文の説明を表示しています。
Learns from DAG execution history to improve future performance. Identifies successful patterns, detects anti-patterns, and provides recommendations. Activate on 'learn patterns', 'execution patterns', 'what worked', 'optimize based on history', 'pattern analysis'. NOT for failure analysis (use dag-failure-analyzer) or performance profiling (use dag-performance-profiler).
日本語の概要は準備中です。原文の説明を表示しています。
Debugs recorded DAG lineage through historical inspection, deterministic decision replay when prerequisites hold, and clearly labeled counterfactual or live re-execution. Activate on "debug DAG", "replay execution", "inspect node state", "compare runs", or "execution diff". NOT for live monitoring (use dag-runtime + websocket-streaming), failure analysis (use dag-ops), or general code debugging.
日本語の概要は準備中です。原文の説明を表示しています。
Provides guidance for authoring Apache Airflow DAGs in Managed Service for Apache Airflow (MSAA; formerly Cloud Composer). Covers environment context discovery, Airflow 2 vs 3 compatibility, authoring best practices, and local/remote validation processes. Use when creating or extending an Airflow DAG. Don't use when authoring Python code unrelated to Airflow DAGs.
日本語の概要は準備中です。原文の説明を表示しています。
Provides guidance for troubleshooting Apache Airflow DAGs (failed DAG runs and task instances) in Managed Service for Apache Airflow (MSAA; formerly Cloud Composer). Use when figuring out reasons for DAG run or task instance failures. Don't use when looking for overall recommendations for Managed Airflow environment performance.
日本語の概要は準備中です。原文の説明を表示しています。
数据工程 — 数据平台从业者的认知操作系统, 覆盖把数据从源系统搬运成可靠 / 可查询 / 可信赖形态供分析 / ML / 数据产品消费的全生命周期 (生成 → 摄取 → 存储 → 转换 → 服务 + 安全/数据管理/DataOps/数据架构/编排/软件工程 六条暗流, Reis & Housley 框架): 摄取与集成 (批 + CDC 变更数据捕获 Debezium + EL 工具 Fivetran/Airbyte/Meltano/dlt + Kafka Connect + schema drift) / 存储与文件表格式 (对象存储数据湖 + 列存 Parquet/ORC/Arrow/Avro + 开放表格式 Apache Iceberg/Delta Lake/Apache Hudi + lakehouse + 分区/compaction) / 转换与建模 (ELT dbt/SQLMesh + Spark + 维度建模 Kimball + Inmon + Data Vault + 大宽表 OBT + 渐变维 SCD + 增量模型 + 语义/指标层) / 编排与工作流 (Apache Airflow/Dagster/Prefect/Mage/Kestra/Apache DolphinScheduler + DAG + 幂等 + 回填 backfill + 数据资产调度) / 批流与实时 (Apache Kafka/Apache Flink/Spark Structured Streaming/Kinesis/Pulsar/Redpanda + Lambda vs Kappa + watermark/窗口/exactly-once + 流式 SQL Materialize/RisingWave + 实时 OLAP ClickHouse/Apache Druid/Apache Pinot/StarRocks/Apache Doris) / 数仓与查询引擎 (Snowflake/BigQuery/Redshift/Databricks SQL/Trino/Presto/DuckDB/Polars + 存算分离 + MPP) / 数据质量测试与可观测性 (dbt tests/Great Expectations/Soda + 数据契约 + Monte Carlo data downtime + 新鲜度/量/schema 异常检测) / 数据治理编目与血缘 (DataHub/Amundsen/OpenMetadata/Unity Catalog + 列级血缘 + PII 分类 + 访问控制 + GDPR) / DataOps 与可靠性 (数据 CI/CD + 转换版本控制 + 环境隔离 + 幂等重处理 + 数据 SLA/SLO + 计算存储 FinOps) / 数据架构范式 (现代数据栈 + lakehouse + data mesh + data fabric + 去中心化 vs 中心化所有权) / 分析工程角色 (dbt 时代连接数据工程与分析的桥) — 不含 数据科学/ML 建模本身 (是下游消费者) / BI 仪表盘制作 (serving 下游) / 数据分析报表为终点 / 'data engineer = 跑 Hadoop 的' 过时窄化 / 通用后端应用开发 (平行学科) (Data Engineering — the cognitive operating system of practitioners who design, build, and operate the data platform: moving data from source systems into reliable, queryable, trustworthy form for analytics / ML / products, covering (a) the data engineering lifecycle (generation → ingestion → storage → transformation → serving, with the undercurrents security / data management / DataOps / data architecture / orchestration / software engineering — Reis & Housley framing), (b) ingestion & integration (batch + CDC change-data-capture with Debezium, EL tools Fivetran / Airbyte / Meltano / dlt, Kafka Connect, API + file + database sources, schema drift handling), (c) storage & file/table formats (object storage data lakes, columnar formats Parquet / ORC / Arrow / Avro, open table formats Apache Iceberg / Delta Lake / Apache Hudi, lakehouse architecture, partitioning / compaction / Z-ordering), (d) transformation & modeling (ELT with dbt / SQLMesh, Spark, dimensional modeling Kimball, Inmon CIF, Data Vault, One Big Table / wide tables, normalization vs denormalization, slowly changing dimensions, incremental models, the semantic / metrics layer), (e) orchestration & workflow (Apache Airflow, Dagster, Prefect, Mage, Kestra, Apache DolphinScheduler, DAGs, idempotency, backfills, data-aware / asset-based scheduling), (f) batch vs streaming & real-time (Apache Kafka, Apache Flink, Spark Structured Streaming, Kinesis / Pulsar / Redpanda, the Lambda vs Kappa debate, watermarks / windowing / exactly-once, streaming SQL Materialize / RisingWave, real-time OLAP ClickHouse / Apache Druid / Apache Pinot / StarRocks / Apache Doris), (g) warehouses & query engines (Snowflake, BigQuery, Redshift, Databricks SQL, Trino / Presto, DuckDB, Polars, decoupled storage & compute, MPP), (h) data quality, testing & observability (dbt tests, Great Expectations, Soda, data contracts, Monte Carlo / data downtime, freshness / volume / schema anomaly detection, unit / integration testing of pipelines), (i) data governance, catalog & lineage (DataHub, Amundsen, OpenMetadata, Unity Catalog, column-level lineage, PII / data classification, access control, GDPR / data privacy), (j) DataOps & reliability (CI/CD for data, version control of transformations, environments, idempotent reprocessing, SLAs / SLOs for data, cost / FinOps for compute & storage), (k) data architecture paradigms (modern data stack, data lakehouse, data mesh, data fabric, decentralized vs centralized ownership), (l) the analytics engineering role (the dbt-era bridge between data engineering and analysis); N
日本語の概要は準備中です。原文の説明を表示しています。
Provides guidance for troubleshooting Apache Airflow DAGs (failed DAG runs and task instances) in Managed Service for Apache Airflow (MSAA; formerly Cloud Composer). Use when figuring out reasons for DAG run or task instance failures. Don't use when looking for overall recommendations for Managed Airflow environment performance.
日本語の概要は準備中です。原文の説明を表示しています。
Provides guidance for authoring Apache Airflow DAGs in Managed Service for Apache Airflow (MSAA; formerly Cloud Composer). Covers environment context discovery, Airflow 2 vs 3 compatibility, authoring best practices, and local/remote validation processes. Use when creating or extending an Airflow DAG. Don't use when authoring Python code unrelated to Airflow DAGs.
日本語の概要は準備中です。原文の説明を表示しています。
Designs human-in-the-loop review points for DAG workflows. Determines what to present to the human, how to collect feedback, and how to route approve/reject/modify decisions back into the DAG. Use when adding approval gates, designing review UX, or handling human feedback in agent workflows. Activate on "human review", "approval gate", "human-in-the-loop", "human gate", "approval workflow", "user review step". NOT for executing human gates at runtime (use dag-runtime with Temporal signals), general UX design, or chatbot conversation design.
日本語の概要は準備中です。原文の説明を表示しています。
Apply practical DAG decomposition, transitive-edge reduction, and reachability indexing to dense dependency graphs. Use when low width and repeated queries justify preprocessing. NOT for cyclic graphs, one-off graph checks, or exact-minimum-chain requirements.
日本語の概要は準備中です。原文の説明を表示しています。
Decompose DAGs into execution chains using width bounds, stratification, virtual nodes, and matching. Use when minimizing workflow lanes in dependency graphs. NOT for cyclic graphs, generic scheduling, or one-off queries with no preprocessing payoff.
日本語の概要は準備中です。原文の説明を表示しています。
Design modern, intuitive DAG/workflow visual editors that feel like LEGO, not LabView
日本語の概要は準備中です。原文の説明を表示しています。
Algorithms for decomposing DAGs into chains for parallel execution and resource optimization
日本語の概要は準備中です。原文の説明を表示しています。
Apply hierarchical decomposition and structural exploitation strategies from DAG theory to complex system design, problem-solving, and abstraction challenges
日本語の概要は準備中です。原文の説明を表示しています。
Analyze hierarchical DAG-like systems by separating undirected substrate from ordering metadata, then classifying diamonds, mixers, shortcuts, and hidden cycle structure. Use for workflow architecture, citation or genealogy DAG comparison, and information-flow diagnosis. NOT for ordinary cycle detection in arbitrary graphs, one-node runtime debugging, or systems with no real ordering constraint.
日本語の概要は準備中です。原文の説明を表示しています。
Use when working with Dagster Deep — dagster deep data orchestration management covering job and asset inventory, run monitoring, schedule and sensor health, partition status, resource configuration, and daemon health checks. Use when investigating run failures, analyzing asset materialization, monitoring schedule ticks, or auditing Dagster configurations.
日本語の概要は準備中です。原文の説明を表示しています。
Use when working with Dagger — dagger CI/CD pipeline management. Covers pipeline-as-code configuration, module management, function execution, caching layers, container builds, and pipeline debugging. Use when managing Dagger pipelines, debugging function execution, inspecting modules, or optimizing container-based CI workflows.
日本語の概要は準備中です。原文の説明を表示しています。
Use when working with Dagster — dagster data orchestration platform management. Covers asset management, pipeline runs, sensor and schedule status, IO manager configuration, partition management, and resource health. Use when checking asset materialization status, investigating run failures, managing schedules/sensors, or analyzing Dagster deployments.
日本語の概要は準備中です。原文の説明を表示しています。
Manage DAG execution lifecycles including start, stop, pause, resume, and cleanup. Activate on 'execution lifecycle', 'stop execution', 'abort DAG', 'graceful shutdown', 'kill process'. NOT for cost estimation, DAG building, or skill selection.
日本語の概要は準備中です。原文の説明を表示しています。
Synthesizes evidence-linked feedback from validation results, confidence evidence, and iteration triggers. Proposes bounded revision guidance for a stated acceptance failure. Activate on 'synthesize feedback', 'improvement suggestions', 'actionable feedback', 'iteration guidance', 'feedback generation'. NOT for iteration detection (use dag-iteration-detector) or convergence tracking (use dag-convergence-monitor).
日本語の概要は準備中です。原文の説明を表示しています。
Design modern, intuitive DAG/workflow visual editors that feel like LEGO, not LabView NOT for runtime DAG admission, graph execution, or claiming usability without user testing.
日本語の概要は準備中です。原文の説明を表示しています。
Matches natural language task descriptions to appropriate skills using semantic similarity, ranks candidates by fit and performance history, and maintains the skill catalog. Use when assigning skills to DAG nodes, searching for the right skill for a task, ranking competing skills, or browsing the skill catalog. Activate on "find skill", "match skill", "which skill", "skill for this task", "skill catalog", "rank skills", "best skill". NOT for executing DAGs (use dag-runtime), creating skills (use skill-architect), or grading skills (use skill-grader).
日本語の概要は準備中です。原文の説明を表示しています。