72 个 · 按相关度排序
Convert Snowflake dbt SQL models to BigQuery through a bulk migration workflow. Preserve Jinja template logic while adapting configuration, explicit type conversions, JSON access, and deduplication patterns, with translated files saved in their directory structure.
中文简介正在准备中。当前显示英语简介。
Design dbt data transformations with clearly organized models, data quality tests, documentation, and incremental processing that updates only part of a dataset. Use it to structure analytics tables and organize a dbt project.
中文简介正在准备中。当前显示英语简介。
Build data quality checks with Great Expectations, dbt tests, and data contracts, which define shared expectations between teams. Check missing values, duplicates, and freshness, then integrate validation into data pipelines and CI/CD.
中文简介正在准备中。当前显示英语简介。
Plan data ingestion and transformation workflows with Apache Airflow, Spark, and dbt. Get guidance on processing changed records, validating data quality, and rerunning failed or historical jobs without creating duplicate results.
中文简介正在准备中。当前显示英语简介。
数据工程 — 数据平台从业者的认知操作系统, 覆盖把数据从源系统搬运成可靠 / 可查询 / 可信赖形态供分析 / ML / 数据产品消费的全生命周期 (生成 → 摄取 → 存储 → 转换 → 服务 + 安全/数据管理/DataOps/数据架构/编排/软件工程 六条暗流, Reis & Housley 框架): 摄取与集成 (批 + CDC 变更数据捕获 Debezium + EL 工具 Fivetran/Airbyte/Meltano/dlt + Kafka Connect + schema drift) / 存储与文件表格式 (对象存储数据湖 + 列存 Parquet/ORC/Arrow/Avro + 开放表格式 Apache Iceberg/Delta Lake/Apache Hudi + lakehouse + 分区/compaction) / 转换与建模 (ELT dbt/SQLMesh + Spark + 维度建模 Kimball + Inmon + Data Vault + 大宽表 OBT + 渐变维 SCD + 增量模型 + 语义/指标层) / 编排与工作流 (Apache Airflow/Dagster/Prefect/Mage/Kestra/Apache DolphinScheduler + DAG + 幂等 + 回填 backfill + 数据资产调度) / 批流与实时 (Apache Kafka/Apache Flink/Spark Structured Streaming/Kinesis/Pulsar/Redpanda + Lambda vs Kappa + watermark/窗口/exactly-once + 流式 SQL Materialize/RisingWave + 实时 OLAP ClickHouse/Apache Druid/Apache Pinot/StarRocks/Apache Doris) / 数仓与查询引擎 (Snowflake/BigQuery/Redshift/Databricks SQL/Trino/Presto/DuckDB/Polars + 存算分离 + MPP) / 数据质量测试与可观测性 (dbt tests/Great Expectations/Soda + 数据契约 + Monte Carlo data downtime + 新鲜度/量/schema 异常检测) / 数据治理编目与血缘 (DataHub/Amundsen/OpenMetadata/Unity Catalog + 列级血缘 + PII 分类 + 访问控制 + GDPR) / DataOps 与可靠性 (数据 CI/CD + 转换版本控制 + 环境隔离 + 幂等重处理 + 数据 SLA/SLO + 计算存储 FinOps) / 数据架构范式 (现代数据栈 + lakehouse + data mesh + data fabric + 去中心化 vs 中心化所有权) / 分析工程角色 (dbt 时代连接数据工程与分析的桥) — 不含 数据科学/ML 建模本身 (是下游消费者) / BI 仪表盘制作 (serving 下游) / 数据分析报表为终点 / 'data engineer = 跑 Hadoop 的' 过时窄化 / 通用后端应用开发 (平行学科) (Data Engineering — the cognitive operating system of practitioners who design, build, and operate the data platform: moving data from source systems into reliable, queryable, trustworthy form for analytics / ML / products, covering (a) the data engineering lifecycle (generation → ingestion → storage → transformation → serving, with the undercurrents security / data management / DataOps / data architecture / orchestration / software engineering — Reis & Housley framing), (b) ingestion & integration (batch + CDC change-data-capture with Debezium, EL tools Fivetran / Airbyte / Meltano / dlt, Kafka Connect, API + file + database sources, schema drift handling), (c) storage & file/table formats (object storage data lakes, columnar formats Parquet / ORC / Arrow / Avro, open table formats Apache Iceberg / Delta Lake / Apache Hudi, lakehouse architecture, partitioning / compaction / Z-ordering), (d) transformation & modeling (ELT with dbt / SQLMesh, Spark, dimensional modeling Kimball, Inmon CIF, Data Vault, One Big Table / wide tables, normalization vs denormalization, slowly changing dimensions, incremental models, the semantic / metrics layer), (e) orchestration & workflow (Apache Airflow, Dagster, Prefect, Mage, Kestra, Apache DolphinScheduler, DAGs, idempotency, backfills, data-aware / asset-based scheduling), (f) batch vs streaming & real-time (Apache Kafka, Apache Flink, Spark Structured Streaming, Kinesis / Pulsar / Redpanda, the Lambda vs Kappa debate, watermarks / windowing / exactly-once, streaming SQL Materialize / RisingWave, real-time OLAP ClickHouse / Apache Druid / Apache Pinot / StarRocks / Apache Doris), (g) warehouses & query engines (Snowflake, BigQuery, Redshift, Databricks SQL, Trino / Presto, DuckDB, Polars, decoupled storage & compute, MPP), (h) data quality, testing & observability (dbt tests, Great Expectations, Soda, data contracts, Monte Carlo / data downtime, freshness / volume / schema anomaly detection, unit / integration testing of pipelines), (i) data governance, catalog & lineage (DataHub, Amundsen, OpenMetadata, Unity Catalog, column-level lineage, PII / data classification, access control, GDPR / data privacy), (j) DataOps & reliability (CI/CD for data, version control of transformations, environments, idempotent reprocessing, SLAs / SLOs for data, cost / FinOps for compute & storage), (k) data architecture paradigms (modern data stack, data lakehouse, data mesh, data fabric, decentralized vs centralized ownership), (l) the analytics engineering role (the dbt-era bridge between data engineering and analysis); N
中文简介正在准备中。当前显示原文说明。
Use when working with Dbt — dbt (data build tool) project management and monitoring. Covers model runs, test results, source freshness, documentation generation, manifest analysis, and dbt Cloud API integration. Use when checking dbt run status, investigating test failures, analyzing model lineage, or managing dbt Cloud jobs.
中文简介正在准备中。当前显示原文说明。
Design and implement systems that collect, transform, validate, and store data for analysis. Covers scheduled and real-time processing, data warehouses, quality checks, monitoring, and cost tuning using tools such as Apache Spark, dbt, and Airflow.
中文简介正在准备中。当前显示英语简介。
dbt Core/Cloud data transformations, testing, documentation, and CI/CD. Activate on: dbt, data transformation, analytics engineering, ref, source, staging model, mart, dbt test. NOT for: orchestration/scheduling (use airflow-dag-orchestrator), data warehouse tuning (use data-warehouse-optimizer).
中文简介正在准备中。当前显示原文说明。
Writes and reviews dbt projects targeting Snowflake: models, schema.yml metadata, sources, tests, macros, and project structure. Use when building or reviewing dbt models, generating dbt YAML, or applying dbt best practices.
中文简介正在准备中。当前显示原文说明。
dbt Core/Cloud data transformations, testing, documentation, and CI/CD. Activate on: dbt, data transformation, analytics engineering, ref, source, staging model, mart, dbt test. NOT for: orchestration/scheduling (use airflow-dag-orchestrator), data warehouse tuning (use data-warehouse-optimizer).
中文简介正在准备中。当前显示原文说明。
Design systems that collect, transform, and store data. Choose between scheduled and continuous processing, generate workflow configurations, validate data quality, and analyze performance using guidance for tools including Python, SQL, Spark, Airflow, dbt, and Kafka.
中文简介正在准备中。当前显示英语简介。
Map dbt `schema.yml` / `properties.yml` models and sources into ktx semantic-layer overlays and column notes. Covers `sources:` vs `models:`, column `data_tests` (not_null, unique, accepted_values, relationships), and how bundle-time writes complement manifest backfill from git sync. Load when the WorkUnit's `skillNames` includes `dbt_ingest` or when raw files are dbt YAML under `models/` / `sources/`.
中文简介正在准备中。当前显示原文说明。
Analytics engineering across data modeling, dbt, transformation, and semantic layers. Use when building dbt models, designing star schemas, writing staging or mart SQL, configuring data tests, or optimizing warehouse queries.
中文简介正在准备中。当前显示原文说明。
Apply dbt transformation patterns in reproducible local data workflows with version-aware APIs, explicit assumptions, and validation. Use when the user chooses dbt transformation patterns or its strengths fit the task.
中文简介正在准备中。当前显示原文说明。
dbt (data build tool) transforms data in your warehouse using SQL SELECT statements. Learn project setup, models, tests, documentation, incremental materializations, and integration with data warehouses like PostgreSQL, BigQuery, and Snowflake.
中文简介正在准备中。当前显示原文说明。
Use when writing or changing analytical SQL or a dbt model — CTE structure, window functions, grain and join correctness, dbt data tests, unit tests, contracts and incremental models, and how to run them
中文简介正在准备中。当前显示原文说明。
Analytics engineering with dbt — models, tests, and modular SQL pipelines — use when building or maintaining dbt projects.
中文简介正在准备中。当前显示原文说明。
Translates Snowflake dbt SQL models to Standardized BigQuery SQL. Handles SQL compilation, Jinja macro placeholder masking, BigQuery Translation Service migration workflows, AST-based config transformations, explicit type casting, JSON extraction standardization, and deduplication. Use when migrating Snowflake dbt pipelines or models to Google Cloud BigQuery. Don't use for generic BigQuery queries or non-Snowflake SQL migrations.
中文简介正在准备中。当前显示原文说明。
Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).
中文简介正在准备中。当前显示原文说明。
Converts ETL pipeline metadata (Informatica IICS, PowerCenter XML, and similar) into clean dbt models, SQL, and YAML rather than a mechanical one-to-one translation. Use when migrating legacy ETL mappings to dbt.
中文简介正在准备中。当前显示原文说明。
Use when working with Dbt Cloud — dbt Cloud management — monitor projects, environments, job runs, model status, and data freshness. Use when reviewing job run health, debugging failed models, inspecting run artifacts, or auditing project configuration.
中文简介正在准备中。当前显示原文说明。
Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).
中文简介正在准备中。当前显示原文说明。
Runs a declarative strategy spec over OHLCV bars with the manifoldbt Rust engine, pairs the fill log into round trips, and emits the eight inputs the backtest-expert skill scores. Use when the user wants to execute a backtest, measure a rule they have described, obtain win rate / average win / average loss / max drawdown from real bars, or feed backtest-expert with measured numbers instead of estimates.
中文简介正在准备中。当前显示原文说明。
Master dbt (data build tool) for analytics engineering with model organization, testing, documentation, and incremental strategies. Use when building data transformations, creating data models, or implementing analytics engineering best practices.
中文简介正在准备中。当前显示原文说明。