「data quality」の検索結果

数据工程 — 数据平台从业者的认知操作系统, 覆盖把数据从源系统搬运成可靠 / 可查询 / 可信赖形态供分析 / ML / 数据产品消费的全生命周期 (生成 → 摄取 → 存储 → 转换 → 服务 + 安全/数据管理/DataOps/数据架构/编排/软件工程 六条暗流, Reis & Housley 框架): 摄取与集成 (批 + CDC 变更数据捕获 Debezium + EL 工具 Fivetran/Airbyte/Meltano/dlt + Kafka Connect + schema drift) / 存储与文件表格式 (对象存储数据湖 + 列存 Parquet/ORC/Arrow/Avro + 开放表格式 Apache Iceberg/Delta Lake/Apache Hudi + lakehouse + 分区/compaction) / 转换与建模 (ELT dbt/SQLMesh + Spark + 维度建模 Kimball + Inmon + Data Vault + 大宽表 OBT + 渐变维 SCD + 增量模型 + 语义/指标层) / 编排与工作流 (Apache Airflow/Dagster/Prefect/Mage/Kestra/Apache DolphinScheduler + DAG + 幂等 + 回填 backfill + 数据资产调度) / 批流与实时 (Apache Kafka/Apache Flink/Spark Structured Streaming/Kinesis/Pulsar/Redpanda + Lambda vs Kappa + watermark/窗口/exactly-once + 流式 SQL Materialize/RisingWave + 实时 OLAP ClickHouse/Apache Druid/Apache Pinot/StarRocks/Apache Doris) / 数仓与查询引擎 (Snowflake/BigQuery/Redshift/Databricks SQL/Trino/Presto/DuckDB/Polars + 存算分离 + MPP) / 数据质量测试与可观测性 (dbt tests/Great Expectations/Soda + 数据契约 + Monte Carlo data downtime + 新鲜度/量/schema 异常检测) / 数据治理编目与血缘 (DataHub/Amundsen/OpenMetadata/Unity Catalog + 列级血缘 + PII 分类 + 访问控制 + GDPR) / DataOps 与可靠性 (数据 CI/CD + 转换版本控制 + 环境隔离 + 幂等重处理 + 数据 SLA/SLO + 计算存储 FinOps) / 数据架构范式 (现代数据栈 + lakehouse + data mesh + data fabric + 去中心化 vs 中心化所有权) / 分析工程角色 (dbt 时代连接数据工程与分析的桥) — 不含 数据科学/ML 建模本身 (是下游消费者) / BI 仪表盘制作 (serving 下游) / 数据分析报表为终点 / 'data engineer = 跑 Hadoop 的' 过时窄化 / 通用后端应用开发 (平行学科) (Data Engineering — the cognitive operating system of practitioners who design, build, and operate the data platform: moving data from source systems into reliable, queryable, trustworthy form for analytics / ML / products, covering (a) the data engineering lifecycle (generation → ingestion → storage → transformation → serving, with the undercurrents security / data management / DataOps / data architecture / orchestration / software engineering — Reis & Housley framing), (b) ingestion & integration (batch + CDC change-data-capture with Debezium, EL tools Fivetran / Airbyte / Meltano / dlt, Kafka Connect, API + file + database sources, schema drift handling), (c) storage & file/table formats (object storage data lakes, columnar formats Parquet / ORC / Arrow / Avro, open table formats Apache Iceberg / Delta Lake / Apache Hudi, lakehouse architecture, partitioning / compaction / Z-ordering), (d) transformation & modeling (ELT with dbt / SQLMesh, Spark, dimensional modeling Kimball, Inmon CIF, Data Vault, One Big Table / wide tables, normalization vs denormalization, slowly changing dimensions, incremental models, the semantic / metrics layer), (e) orchestration & workflow (Apache Airflow, Dagster, Prefect, Mage, Kestra, Apache DolphinScheduler, DAGs, idempotency, backfills, data-aware / asset-based scheduling), (f) batch vs streaming & real-time (Apache Kafka, Apache Flink, Spark Structured Streaming, Kinesis / Pulsar / Redpanda, the Lambda vs Kappa debate, watermarks / windowing / exactly-once, streaming SQL Materialize / RisingWave, real-time OLAP ClickHouse / Apache Druid / Apache Pinot / StarRocks / Apache Doris), (g) warehouses & query engines (Snowflake, BigQuery, Redshift, Databricks SQL, Trino / Presto, DuckDB, Polars, decoupled storage & compute, MPP), (h) data quality, testing & observability (dbt tests, Great Expectations, Soda, data contracts, Monte Carlo / data downtime, freshness / volume / schema anomaly detection, unit / integration testing of pipelines), (i) data governance, catalog & lineage (DataHub, Amundsen, OpenMetadata, Unity Catalog, column-level lineage, PII / data classification, access control, GDPR / data privacy), (j) DataOps & reliability (CI/CD for data, version control of transformations, environments, idempotent reprocessing, SLAs / SLOs for data, cost / FinOps for compute & storage), (k) data architecture paradigms (modern data stack, data lakehouse, data mesh, data fabric, decentralized vs centralized ownership), (l) the analytics engineering role (the dbt-era bridge between data engineering and analysis); N

日本語の概要は準備中です。原文の説明を表示しています。

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, real-time streaming, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, Flink, Kinesis, and modern data stack. Includes data modeling, pipeline orchestration, data quality, streaming quality monitoring, and DataOps. Use when designing data architectures, building batch or streaming data pipelines, optimizing data workflows, or implementing data governance.

日本語の概要は準備中です。原文の説明を表示しています。

Data pipeline specialist for ETL design, data quality, CDC patterns, and batch/stream processingUse when "data pipeline, etl, cdc, data quality, batch processing, stream processing, data transformation, data warehouse, data lake, data validation, data-engineering, etl, cdc, batch, streaming, data-quality, dbt, airflow, dagster, data-pipeline, ml-memory" mentioned.

日本語の概要は準備中です。原文の説明を表示しています。

Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, implementing data governance, or troubleshooting data issues.

日本語の概要は準備中です。原文の説明を表示しています。

Validate data quality in CSV, JSON, and database exports by checking for missing values, type mismatches, duplicates, outliers, and schema violations. Use when building ETL pipelines, auditing data imports, checking data freshness, or ensuring data contracts between teams. Trigger words: data quality, validation, null values, duplicates, schema check, data contract, ETL, pipeline, data drift.

日本語の概要は準備中です。原文の説明を表示しています。

OpenLineage, DataHub, Marquez for data lineage tracking and impact analysis. Activate on: data lineage, OpenLineage, DataHub, Marquez, impact analysis, data catalog, column lineage, data discovery. NOT for: data quality validation (use data-quality-guardian), dbt documentation (use dbt-analytics-engineer).

日本語の概要は準備中です。原文の説明を表示しています。

OpenLineage, DataHub, Marquez for data lineage tracking and impact analysis. Activate on: data lineage, OpenLineage, DataHub, Marquez, impact analysis, data catalog, column lineage, data discovery. NOT for: data quality validation (use data-quality-guardian), dbt documentation (use dbt-analytics-engineer).

日本語の概要は準備中です。原文の説明を表示しています。

Prepare high-quality datasets for LLM fine-tuning with filtering, deduplication, augmentation, and RLHF data formatting. Activate on: fine-tuning data, training data curation, RLHF dataset, data quality filtering, SFT dataset. NOT for: model training infrastructure (ai-engineer), prompt engineering without fine-tuning (prompt-engineer).

日本語の概要は準備中です。原文の説明を表示しています。

Build training datasets for LLM specialization from production data, frontier model distillation, and synthetic bootstrapping. Use when formatting production logs into SFT data, distilling from frontier APIs, or preparing data for fine-tuning. Covers JSONL formatting, data quality validation, deduplication, and train/eval splitting.

日本語の概要は準備中です。原文の説明を表示しています。

Prepare high-quality datasets for LLM fine-tuning with filtering, deduplication, augmentation, and RLHF data formatting. Activate on: fine-tuning data, training data curation, RLHF dataset, data quality filtering, SFT dataset. NOT for: model training infrastructure (ai-engineer), prompt engineering without fine-tuning (prompt-engineer).

日本語の概要は準備中です。原文の説明を表示しています。

Run exploratory data analysis on a dataset and produce a structured report. Use when the user says "explore this dataset", "EDA on X", "analyze this data", "what's in this dataset", "summarize this data", "first look at the data", "understand this dataset before modeling", "data quality check", "describe this dataframe", or wants to understand a new dataset before building models or dashboards.

日本語の概要は準備中です。原文の説明を表示しています。

Router for the 101 SfSkills `data` skill packages. Data model, data movement and data quality: migrations, bulk loads, query optimisation, deduplicating at volume, archival. Ordinary-volume duplicate cleanup and prevention use salesforce-admin; come here for hundreds-of-thousands+ dedup or third-party tools. LDV architecture uses salesforce-architect. Use when the request mentions data model, data migration, data load, Data Loader, Bulk API, external id, deduplication at volume, archival, SOSL, cross-object search, sandbox seed data, SandboxPostCopy, native Data Seeding. Finds and opens the exact skill package to read; it does not contain the guidance itself.

日本語の概要は準備中です。原文の説明を表示しています。

Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).

日本語の概要は準備中です。原文の説明を表示しています。

Great Expectations, dbt tests, anomaly detection, and data contracts for data quality. Activate on: data quality, data validation, Great Expectations, data contract, anomaly detection, SLA, freshness check, schema validation. NOT for: dbt model structure (use dbt-analytics-engineer), schema evolution (use schema-evolution-manager).

日本語の概要は準備中です。原文の説明を表示しています。

Data leadership advisor on data strategy, governance, quality, and platform decisions. Use when defining a data strategy, scoring data maturity, auditing data governance, evaluating a data platform, or designing the data org.

日本語の概要は準備中です。原文の説明を表示しています。

Data governance strategy, quality validation rules, and data dictionary management for vehicle insurance platform. Use when defining data quality standards, implementing validation rules, managing field mappings, resolving data conflicts, or establishing data governance processes. Covers data cleaning standards, quality metrics, and mapping management.

日本語の概要は準備中です。原文の説明を表示しています。

Marketing analytics and data-driven optimization for B2B SaaS campaigns. Use this skill when analyzing marketing performance, optimizing conversion rates, running A/B tests, conducting customer research, creating marketing dashboards, building attribution models, analyzing funnel metrics, or storytelling with data. Also triggers on: marketing ops, performance analytics, CRO, A/B testing, data storytelling, customer insights, tracking plan, event taxonomy, UTM taxonomy, GA4 audit, consent mode, instrumentation contract, sample size calculation, minimum detectable effect, test duration, not enough traffic to A/B test, statistical significance, Bayesian vs frequentist, sample ratio mismatch, experiment feasibility, martech stack, marketing tech stack audit, tool consolidation, build vs buy, vendor evaluation, shadow IT tools, ESP/CRM/platform migration, switching cost, cutover plan, suppression list migration, data portability, customer interviews, survey design, sampling frame, non-response bias, how many people should I interview, thematic saturation, voice of customer, jobs-to-be-done, buyer persona research, is this finding real, data enrichment, which enrichment provider should we buy, waterfall enrichment, contact data quality, match rate, our list data is bad, email verification, catch-all domain, accept-all, bounce rate from bad data, B2B data decay, how often should we re-verify contacts, ZoomInfo vs Apollo vs Clearbit evaluation, cost per usable record, where did this list come from, GDPR Article 14 notice, purchased list compliance, suppression screening, TAM list building, demand planning, marketing plan, how many leads do we need, what pipeline target should marketing carry, funnel model, top-down vs bottom-up plan, plan of record, annual marketing planning, quarterly planning, pipeline coverage we have to create, sales cycle lag, our leads cannot close this quarter, carry-in pipeline, scenario planning, conservative moderate aggressive forecast, stretch target, reforecast, re-forecast triggers, assumption register, capacity ceiling, addressable audience ceiling, marginal cost per lead, blended CPL is wrong, we missed plan and nobody knows why, AI policy for the marketing team, AI governance, AI acceptable use policy, which AI tools are we allowed to use, can I paste customer data into ChatGPT, shadow AI, AI use register, human review of AI output, when can we stop reviewing every AI draft, can the agent send on its own, the AI made a mistake now what, pull back AI autonomy, AI evals, our AI vendor changed the model, AI output went out wrong, prompt injection, can our AI read prospect replies safely, lethal trifecta, the AI leaked internal notes, turn off auto-generated ad assets, auto-applied recommendations, AI literacy training, EU AI Act Article 4, duplicate records in the CRM, dedupe before import, which value wins in a merge, CRM cleanup.

日本語の概要は準備中です。原文の説明を表示しています。

Turns reported data problems (messages, tickets, meeting notes, complaints) into one DRAFT issue log: one row per issue with the symptom as reported, dataset and fields named, reproduction steps, affected reports, first seen date, reporters, a suggested owner, and blank severity, root cause and status cells for the owner. Never asserts a root cause, severity or fix. Use when the user asks to "log these data issues", "turn these complaints about the numbers into an issue list", "build the data quality backlog", "which reports are hit by this data problem" or "write up the reproduction steps for this data bug". Do not use for exploring or profiling a dataset, use dataset-insight-pack instead; for an IT incident write-up, use incident-postmortem-drafter; for tracking fix actions once assigned, use corrective-action-tracker. Drafts for human review; never approves, authorises or signs off.

日本語の概要は準備中です。原文の説明を表示しています。

Insurance-industry App-in-Skill for read-only data governance, backed by an operator-provisioned Busabase workspace (one Drive node for the file drive plus four Bases for QA pairs, featured information, insurer notices, and user feedback) and trusted export/restore/PDF-text-backfill scripts. Use when the user invokes $kelly-insure-data or /kelly-insure-data, wants an insurance data workspace with UI, needs to review insurance files, metadata completeness, QA pairs, featured information, insurer notices, or user feedback, wants to back up or restore a Kelly Insure Data Busabase workspace from local PDFs, or wants Busabase Drive/Base data surfaced for data quality review and ongoing data governance.

日本語の概要は準備中です。原文の説明を表示しています。

Use when implementing data governance frameworks, building data catalogs, establishing data lineage, defining data quality rules, or setting up data stewardship programs - covers metadata management, data quality, and complianceUse when ", " mentioned.

日本語の概要は準備中です。原文の説明を表示しています。

Audit existing data cleaning code — find missing validation, silent data loss, and quality gaps. Use when asked to "audit our data cleaning", "are we losing data silently", or "find data quality gaps".

日本語の概要は準備中です。原文の説明を表示しています。

Audit data quality across six DAMA DMBOK dimensions — Completeness, Accuracy, Consistency, Timeliness, Validity, Uniqueness. Produces a scored data quality report with issue log and remediation plan. Use when profiling datasets, onboarding new data sources, or building data quality gates.

日本語の概要は準備中です。原文の説明を表示しています。

Evaluate ENCODE experiment quality using standard metrics and audit flags. Use when the user asks about data quality, wants to filter for high-quality experiments, needs to interpret quality metrics (FRiP, NSC, RSC, NRF, IDR, TSS enrichment, fragment size), wants to understand ENCODE audit warnings, needs to compare quality across experiments, or is deciding whether data is usable for their analysis. Also use when the user mentions QC, quality control, or data filtering.

日本語の概要は準備中です。原文の説明を表示しています。

Build QA quality dashboards, calculate defect metrics, analyze escape rates, and produce quality trend reports. Use this skill whenever a QA lead or manager needs to measure and communicate quality outcomes. Trigger on phrases like "quality metrics", "defect density", "escape rate", "bug trends", "quality dashboard", "sprint quality report", "test coverage metrics", "QA KPIs", "how is quality trending", "defect leakage", "production bugs this quarter", "mean time to detect", "quality scorecard", or any request to quantify, visualize, or report on the quality of a product or testing process. Also trigger when a manager asks "how is QA performing?" or an engineering lead asks "what does our defect data tell us?".

日本語の概要は準備中です。原文の説明を表示しています。