Reference catalog of data-quality conventions - when to choose dbt-tests vs Great Expectations vs Soda, column-level vs table-level coverage, severity tiering, SLA and freshness conventions, and common anti-patterns to avoid. Use when designing coverage for a new data product or auditing an existing one.
中文简介正在准备中。当前显示原文说明。
实测评分 未实测
查看安装方法安装前可以查看交给 AI 代理的指令内容。
A reference catalog for how to design data-quality coverage. Pairs
with the engine-specific skills
(dbt-testing,
great-expectations,
soda-checks) - those tell you the how
of running checks; this tells you which checks and where.
Use this matrix to pick an engine before authoring any checks. Mixing
engines is fine in a large platform - the data-quality-gate
skill exists precisely to reconcile their outputs.
| Use case | Preferred engine | Why |
|---|---|---|
| Repo is a dbt project; assertions live next to models | dbt-tests | Tests run as part of dbt build, share the project's adapter, and live in the same schema.yml as the column docs. |
| Python ELT, programmatic suite generation, runtime data connectors | Great Expectations | First-class Pandas / SQL / Spark batches; suites composable in Python. |
| YAML-only checks against a SQL warehouse, cross-team observability via Soda Cloud | Soda | SodaCL is a focused DSL; no Python or dbt required. |
| Mix of warehouse + non-warehouse sources | dbt + GX or Soda + GX | Use dbt or Soda for warehouse coverage; GX for non-warehouse (file uploads, Spark jobs, ad-hoc Pandas). |
| Brand-new project, no preference | dbt-tests if you already have dbt; otherwise Soda for YAML simplicity | Lowest cognitive overhead. |
Anti-pattern: running all three engines on the same dataset for the same checks "for redundancy" - duplicate checks shift the failure cost to humans triaging three CI signals for the same root cause.
Authoring tip: start at the column level for the keys and required fields, then add table-level invariants. A reasonable starter set for a new data product:
| Layer | Check | Justification |
|---|---|---|
| Column (PK) | not_null, unique | Catches the largest classes of ingestion bugs. |
| Column (FK) | relationships to the referenced table | Catches out-of-order loads. |
| Column (enum) | accepted_values for low-cardinality strings | Catches new states added upstream without coordination. |
| Column (numeric) | range / between for business-bounded columns | Catches arithmetic errors and unit mismatches. |
| Column (date) | not_null on required timestamps | Catches dropped fields. |
| Table | row_count > 0 (and an upper bound if known) | Catches catastrophic ingestion drop or runaway duplicate. |
| Table | freshness on the load-time timestamp | Catches "pipeline didn't run" - the most common silent failure. |
For numeric columns, prefer business-bounded ranges (e.g. discount 0 - 100 %) over distribution-bounded ranges (e.g. mean ± 3σ). Statistical ranges drift with seasonality and produce false positives at scale; the shop should treat distribution monitoring as a separate concern from data-quality assertions.
Three tiers cover most needs:
| Tier | Behavior on failure | Use for |
|---|---|---|
error | Block the pipeline / CI | Invariants the business depends on (PK uniqueness, FK integrity, required fields). |
warn | Surface in the report, do not block | Distribution shifts, soft constraints, new unfamiliar checks during ramp-up. |
info | Log only, never alert | Coverage telemetry (e.g. "70 % of customer rows had email populated"). |
In dbt, severity comes from a severity: config block; in GX, from
suite-level metadata; in SodaCL, from warn: / fail: blocks for
schema and from the alert configuration syntax for other checks. Each
engine's specifics are in its SKILL.md.
Anti-pattern: every check at error. A pipeline that blocks on a
single distribution drift becomes "the check that always fails" within
a quarter; the team eventually disables the gate entirely. Reserve
error for invariants the business will actually halt to fix.
A freshness check is the highest-leverage assertion in most pipelines - it catches the "pipeline didn't run" failure mode that no row-level check can detect (because there are no rows to check).
Conventions:
freshness < 2d; an hourly pipeline gets < 2h.
This avoids false alarms from a single late run while still catching
a stuck pipeline.error-severity by convention - a stuck pipeline blocks
every downstream business decision, so a noisy alert is the right
behavior.not_null_orders_email (dbt-style) or
missing_count(email) = 0 (SodaCL) is self-documenting; assertion_42
is not.meta: block under the column or
model with owner: @team-handle. GX: suite-level metadata. SodaCL:
the dataset's meta: block. Failure routing uses these to send
each failure to its owner.| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Hard-coded distributions ("revenue between $X and $Y") | Drifts with seasonality; ages out within months | Bound with a moving window or replace with a separate distribution-monitoring tool. |
not_null on every column | Some columns are legitimately nullable; floods the failure list with non-actionable noise | Annotate nullable columns explicitly in schema.yml and skip the test. |
| One mega-check that asserts an entire row at once | Hard to triage; a single failure does not tell you which field broke | Split into one check per invariant; aggregate at the gate level via the data-quality-gate. |
| Reaching into upstream raw data inside a check | Couples the assertion to source schema; breaks on every upstream rename | Test the curated model output, not the raw source. |
| Same check authored in dbt and GX and Soda | Three failure signals for one root cause; triage cost triples | Pick one engine per check; use the gate skill to unify cross-engine coverage. |
Severity error on a freshly added check | First false positive disables the gate by social convention | Land new checks at warn for two cycles, then promote once they're seen to be stable. |
A check is healthy when it fails roughly once per quarter on a real issue. Two failure modes warrant retirement:
还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Builds a CI gate that fails the build on **new** WCAG / a11y violations introduced by a PR while grandfathering pre-existing violations on a per-rule / per-page baseline. Aggregates verdicts from axe-core / pa11y / Lighthouse a11y / WAVE / IBM Equal Access scans. Use when a project has accumulated a11y debt and a strict "zero violations" gate would block every PR - the ratchet pattern lets the team ship while preventing regressions.
中文简介正在准备中。当前显示原文说明。
Workflow skill that builds an A/B-test validity checklist from an experiment proposal, walking the canonical design-correctness gates - pre-registered OEC/power/guardrails, randomization unit + SRM check, assignment integrity, telemetry, peeking discipline, novelty/primacy, post-experiment SRM re-check - into a per-experiment checklist + sign-off form. Use when launching, auditing, or governing an experiment. For pitfall mechanics (guardrails, peeking) see experiment-results-interpreter's references; to read an already-valid result use experiment-results-interpreter; for per-SDK harness tests use experiment-sdk-testing - this gates DESIGN, not SDK code.
中文简介正在准备中。当前显示原文说明。
The single install-flow skill: the reference contract for the PWA install flow (installability gate fields, `beforeinstallprompt` handshake, per-platform paths - Android WebAPK / iOS Share menu / Firefox no-op - and the `display-mode` post-install signal, in references/install-flow-reference.md) plus the build-an-X workflow that emits the Add-to-Home-Screen suite. Walks the four-stage timeline, emitting one test per gate cell per [web.dev/articles/install-criteria][install-criteria], the deferred-prompt → `prompt()` → `userChoice` chain per [web.dev/articles/customize-install][customize-install], the iOS Safari manual-metadata branch (`apple-touch-icon`) per [web.dev/learn/pwa/installation][learn-pwa], and the post-install `(display-mode: standalone)` MQ assertion. Output: a Playwright spec with per-stage cells plus a coverage matrix. Use when a PWA's manifest, icons, or install handler change, when install conversion drops at an unknown stage, or when triaging a flaky install assertion.
中文简介正在准备中。当前显示原文说明。
Build-an-X workflow that uses an LLM to map existing tests to spec sections - given a spec doc + the test suite, the LLM identifies which tests cover which sections, surfaces uncovered sections (gap), and recommends specific tests to add. Output is a coverage matrix per spec ID. Scope is mapping tests that already exist and naming the gaps, not authoring tests for new acceptance criteria. Use when a spec doc and a test suite both exist but nobody can say which requirements are actually covered - before a release sign-off, an audit, or a decision about where to spend the next round of test effort.
中文简介正在准备中。当前显示原文说明。
Generates tests from natural-language specs (acceptance criteria, user stories, requirements) using an LLM, with confidence scoring per test case (LLM self-assessment plus heuristics: assertion quality, naming, completeness), batching uncertain cases for human review, and integration with the team's existing test framework. Use when the user asks to generate unit tests from acceptance criteria, convert user stories to test cases, automate test creation from requirements, or augment a spec-driven test suite with AI-generated stubs that are then curated before merge.
中文简介正在准备中。当前显示原文说明。
Configures Allure Report (test-runner adapter install, `allure-results` directory wiring, `categories.json` for failure classification, `history-trend.json` retention via the copy-history-between-runs pattern), runs the Allure CLI to convert `allure-results` to a static HTML site, and uploads the report as a CI artifact. Use when the team needs richer test reporting than JUnit XML - step-level attachments, per-test history, retry tracking, and severity / epic / feature labeling across framework-agnostic adapters (pytest, Jest, JUnit, TestNG, NUnit, Mocha). As a rich static HTML report generator, it is the open-source alternative to the sunset ExtentReports (JVM/.NET per-test HTML narrative); for hosted cross-run flakiness analytics rather than a static per-run report use currents-integration.
中文简介正在准备中。当前显示原文说明。