Run Deepchecks suites (data integrity, train-test validation, model evaluation) on tabular / NLP / vision data + models. Pass `result.passed_conditions()` to CI to gate on regressions; the same checks run during research, CI, and production monitoring per the Deepchecks lifecycle posture. Use before training to catch train-test leakage and data-integrity defects in a tabular, NLP, or vision dataset, and to re-run the same suite on production samples to detect drift.
中文简介正在准备中。当前显示原文说明。
实测评分 未实测
查看安装方法安装前可以查看交给 AI 代理的指令内容。
Deepchecks is "a holistic open-source solution for all of your AI & ML validation needs" per the Deepchecks welcome. Validates data integrity, train-test splits, model evaluation, end-to-end model development from research through production.
pip install deepchecks
Per the Deepchecks welcome page.
For tabular models:
from deepchecks.tabular import Dataset
train_ds = Dataset(
train_df,
label="target",
cat_features=CATEGORICAL_COLUMNS,
)
test_ds = Dataset(
test_df,
label="target",
cat_features=CATEGORICAL_COLUMNS,
)
cat_features matters for distribution checks. The Vision and NLP
APIs differ - see the Deepchecks welcome section linking
quickstarts for each data type.
from deepchecks.tabular.suites import data_integrity
integrity = data_integrity()
result = integrity.run(train_ds)
result.save_as_html("data_integrity.html")
Catches: duplicate rows, missing values, mixed types, conflicting labels, single-value features, string mismatches.
from deepchecks.tabular.suites import train_test_validation
validation = train_test_validation()
result = validation.run(train_ds, test_ds)
result.save_as_html("train_test_validation.html")
Catches: target drift, feature drift, train-test data leakage, label imbalance, dataset size mismatch.
from deepchecks.tabular.suites import model_evaluation
evaluation = model_evaluation()
result = evaluation.run(train_ds, test_ds, model)
result.save_as_html("model_evaluation.html")
if not result.passed_conditions():
raise SystemExit("Deepchecks model evaluation failed")
Catches: performance regression vs baseline, weak segments, calibration issues, prediction drift between train and test.
from deepchecks.tabular.checks import FeatureDrift
check = FeatureDrift().add_condition_drift_score_less_than(
max_allowed_categorical_score=0.2,
max_allowed_numeric_score=0.1,
)
result = check.run(train_ds, test_ds)
if not result.passed_conditions():
print(result.value)
raise SystemExit("FeatureDrift failed threshold")
Each check has add_condition_* methods; chain them for per-check
gating.
- name: Deepchecks suite
run: |
python ml/deepchecks_suite.py
- name: Upload Deepchecks reports
uses: actions/upload-artifact@v4
with:
name: deepchecks-reports
path: "*.html"
| Anti-pattern | Why it fails | Fix |
|---|---|---|
Skip cat_features | All categorical checks misfire | Always specify (Step 2) |
Skip data_integrity suite | Train on leaky / dup-heavy data | Run before train_test_validation (Step 3) |
| Block CI on every check | Hundreds of warning conditions; team disables | Define per-check thresholds (Step 6); gate Critical only |
| Re-run on the SAME test split each PR | Fixed split → fixed results; no drift detection | Use rolling/cross-validation splits |
| Reuse training data as "current" for production monitoring | Always passes drift; blind to real drift | Use real production samples |
add_condition_* methods (Step 6).还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Builds a CI gate that fails the build on **new** WCAG / a11y violations introduced by a PR while grandfathering pre-existing violations on a per-rule / per-page baseline. Aggregates verdicts from axe-core / pa11y / Lighthouse a11y / WAVE / IBM Equal Access scans. Use when a project has accumulated a11y debt and a strict "zero violations" gate would block every PR - the ratchet pattern lets the team ship while preventing regressions.
中文简介正在准备中。当前显示原文说明。
Workflow skill that builds an A/B-test validity checklist from an experiment proposal, walking the canonical design-correctness gates - pre-registered OEC/power/guardrails, randomization unit + SRM check, assignment integrity, telemetry, peeking discipline, novelty/primacy, post-experiment SRM re-check - into a per-experiment checklist + sign-off form. Use when launching, auditing, or governing an experiment. For pitfall mechanics (guardrails, peeking) see experiment-results-interpreter's references; to read an already-valid result use experiment-results-interpreter; for per-SDK harness tests use experiment-sdk-testing - this gates DESIGN, not SDK code.
中文简介正在准备中。当前显示原文说明。
The single install-flow skill: the reference contract for the PWA install flow (installability gate fields, `beforeinstallprompt` handshake, per-platform paths - Android WebAPK / iOS Share menu / Firefox no-op - and the `display-mode` post-install signal, in references/install-flow-reference.md) plus the build-an-X workflow that emits the Add-to-Home-Screen suite. Walks the four-stage timeline, emitting one test per gate cell per [web.dev/articles/install-criteria][install-criteria], the deferred-prompt → `prompt()` → `userChoice` chain per [web.dev/articles/customize-install][customize-install], the iOS Safari manual-metadata branch (`apple-touch-icon`) per [web.dev/learn/pwa/installation][learn-pwa], and the post-install `(display-mode: standalone)` MQ assertion. Output: a Playwright spec with per-stage cells plus a coverage matrix. Use when a PWA's manifest, icons, or install handler change, when install conversion drops at an unknown stage, or when triaging a flaky install assertion.
中文简介正在准备中。当前显示原文说明。
Build-an-X workflow that uses an LLM to map existing tests to spec sections - given a spec doc + the test suite, the LLM identifies which tests cover which sections, surfaces uncovered sections (gap), and recommends specific tests to add. Output is a coverage matrix per spec ID. Scope is mapping tests that already exist and naming the gaps, not authoring tests for new acceptance criteria. Use when a spec doc and a test suite both exist but nobody can say which requirements are actually covered - before a release sign-off, an audit, or a decision about where to spend the next round of test effort.
中文简介正在准备中。当前显示原文说明。
Generates tests from natural-language specs (acceptance criteria, user stories, requirements) using an LLM, with confidence scoring per test case (LLM self-assessment plus heuristics: assertion quality, naming, completeness), batching uncertain cases for human review, and integration with the team's existing test framework. Use when the user asks to generate unit tests from acceptance criteria, convert user stories to test cases, automate test creation from requirements, or augment a spec-driven test suite with AI-generated stubs that are then curated before merge.
中文简介正在准备中。当前显示原文说明。
Configures Allure Report (test-runner adapter install, `allure-results` directory wiring, `categories.json` for failure classification, `history-trend.json` retention via the copy-history-between-runs pattern), runs the Allure CLI to convert `allure-results` to a static HTML site, and uploads the report as a CI artifact. Use when the team needs richer test reporting than JUnit XML - step-level attachments, per-test history, retry tracking, and severity / epic / feature labeling across framework-agnostic adapters (pytest, Jest, JUnit, TestNG, NUnit, Mocha). As a rich static HTML report generator, it is the open-source alternative to the sunset ExtentReports (JVM/.NET per-test HTML narrative); for hosted cross-run flakiness analytics rather than a static per-run report use currents-integration.
中文简介正在准备中。当前显示原文说明。