Compute group fairness metrics (selection rate, demographic parity, equalized odds) per sensitive feature with `MetricFrame`, then mitigate disparities using Reductions algorithms (`ExponentiatedGradient` with constraint = `DemographicParity`/`EqualizedOdds`). Wire group-disaggregated assertions into the model-evaluation gate. Use when a model's decisions affect people and a stakeholder, auditor, or regulation (ECOA, GDPR Art. 22, EU AI Act high-risk) requires evidence of per-group outcomes, or when someone reports the model treats a specific group worse.
中文简介正在准备中。当前显示原文说明。
实测评分 未实测
查看安装方法安装前可以查看交给 AI 代理的指令内容。
Fairlearn provides "Metrics - Tools to assess which groups are
negatively impacted and compare models across fairness and accuracy
dimensions" and "Algorithms - Techniques to mitigate unfairness"
per the Fairlearn quickstart. Two primitives: MetricFrame (group
disaggregation) + Reductions (ExponentiatedGradient,
ThresholdOptimizer).
pip install fairlearn
# OR
conda install -c conda-forge fairlearn
Per the Fairlearn quickstart.
from fairlearn.metrics import MetricFrame
from sklearn.metrics import accuracy_score
from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(min_samples_leaf=10, max_depth=4)
classifier.fit(X, y_true)
y_pred = classifier.predict(X)
mf = MetricFrame(
metrics=accuracy_score,
y_true=y_true,
y_pred=y_pred,
sensitive_features=sex,
)
print(mf.by_group)
print(f"Disparity (max-min): {mf.difference()}")
Per the Fairlearn quickstart. sensitive_features can be a Series
or a 2-D array for intersectional analysis (sex × race).
from fairlearn.metrics import selection_rate
sr = MetricFrame(
metrics=selection_rate,
y_true=y_true,
y_pred=y_pred,
sensitive_features=sex,
)
print(sr.by_group)
# Demographic Parity Difference (DPD)
print(f"DPD: {sr.difference()}")
DPD = max group selection rate − min group selection rate. Industry guidance often cites the 80% rule (selection rate ratio ≥ 0.8 between groups) as a soft threshold; consult legal counsel for binding thresholds in your jurisdiction.
from fairlearn.metrics import (
true_positive_rate,
false_positive_rate,
MetricFrame,
)
mf = MetricFrame(
metrics={
"TPR": true_positive_rate,
"FPR": false_positive_rate,
"selection_rate": selection_rate,
},
y_true=y_true,
y_pred=y_pred,
sensitive_features=sex,
)
print(mf.by_group)
Equalized Odds requires both TPR and FPR to be equal across groups - stricter than Demographic Parity.
from fairlearn.reductions import DemographicParity, ExponentiatedGradient
constraint = DemographicParity()
mitigator = ExponentiatedGradient(classifier, constraint)
mitigator.fit(X, y_true, sensitive_features=sex)
y_pred_mitigated = mitigator.predict(X)
Per the Fairlearn quickstart: this approach significantly reduces
selection-rate differences while maintaining accuracy. Other
constraints: EqualizedOdds, TruePositiveRateParity,
FalsePositiveRateParity.
from fairlearn.postprocessing import ThresholdOptimizer
postprocess = ThresholdOptimizer(
estimator=classifier,
constraints="demographic_parity",
prefit=True,
)
postprocess.fit(X, y_true, sensitive_features=sex)
y_pred_pp = postprocess.predict(X, sensitive_features=sex)
Per the Fairlearn postprocessing guide, Fairlearn currently supports
one postprocessing technique, ThresholdOptimizer. Cheaper than
retraining; trades model output for per-group threshold adjustment.
def assert_fairness(y_true, y_pred, sensitive, max_dpd=0.10):
sr = MetricFrame(
metrics=selection_rate,
y_true=y_true,
y_pred=y_pred,
sensitive_features=sensitive,
)
dpd = sr.difference()
if dpd > max_dpd:
raise AssertionError(
f"Demographic Parity Difference {dpd:.3f} exceeds budget {max_dpd}"
)
assert_fairness(y_true, y_pred, sex, max_dpd=0.10)
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Compute aggregate accuracy only | Hides group disparities | Always use MetricFrame (Step 2) |
| Choose Demographic Parity for all problems | DP can be inappropriate when base rates legitimately differ across groups | Match constraint to legal/ethical context: DP, EO, EOD, EOP |
| Mitigate via training data resampling alone | Doesn't generalize to new data; brittle | Use Reductions (Step 5) or post-processing (Step 6) |
| Single sensitive attribute (e.g., sex only) | Misses intersectional disparities (Black women) | Pass 2-D sensitive_features for intersection (Step 2) |
| Hard-code 80% rule globally | Not legally binding everywhere; not appropriate for all metrics | Tune max_dpd per use case + legal counsel; use waiver template if scope-exclusion needed |
还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Builds a CI gate that fails the build on **new** WCAG / a11y violations introduced by a PR while grandfathering pre-existing violations on a per-rule / per-page baseline. Aggregates verdicts from axe-core / pa11y / Lighthouse a11y / WAVE / IBM Equal Access scans. Use when a project has accumulated a11y debt and a strict "zero violations" gate would block every PR - the ratchet pattern lets the team ship while preventing regressions.
中文简介正在准备中。当前显示原文说明。
Workflow skill that builds an A/B-test validity checklist from an experiment proposal, walking the canonical design-correctness gates - pre-registered OEC/power/guardrails, randomization unit + SRM check, assignment integrity, telemetry, peeking discipline, novelty/primacy, post-experiment SRM re-check - into a per-experiment checklist + sign-off form. Use when launching, auditing, or governing an experiment. For pitfall mechanics (guardrails, peeking) see experiment-results-interpreter's references; to read an already-valid result use experiment-results-interpreter; for per-SDK harness tests use experiment-sdk-testing - this gates DESIGN, not SDK code.
中文简介正在准备中。当前显示原文说明。
The single install-flow skill: the reference contract for the PWA install flow (installability gate fields, `beforeinstallprompt` handshake, per-platform paths - Android WebAPK / iOS Share menu / Firefox no-op - and the `display-mode` post-install signal, in references/install-flow-reference.md) plus the build-an-X workflow that emits the Add-to-Home-Screen suite. Walks the four-stage timeline, emitting one test per gate cell per [web.dev/articles/install-criteria][install-criteria], the deferred-prompt → `prompt()` → `userChoice` chain per [web.dev/articles/customize-install][customize-install], the iOS Safari manual-metadata branch (`apple-touch-icon`) per [web.dev/learn/pwa/installation][learn-pwa], and the post-install `(display-mode: standalone)` MQ assertion. Output: a Playwright spec with per-stage cells plus a coverage matrix. Use when a PWA's manifest, icons, or install handler change, when install conversion drops at an unknown stage, or when triaging a flaky install assertion.
中文简介正在准备中。当前显示原文说明。
Build-an-X workflow that uses an LLM to map existing tests to spec sections - given a spec doc + the test suite, the LLM identifies which tests cover which sections, surfaces uncovered sections (gap), and recommends specific tests to add. Output is a coverage matrix per spec ID. Scope is mapping tests that already exist and naming the gaps, not authoring tests for new acceptance criteria. Use when a spec doc and a test suite both exist but nobody can say which requirements are actually covered - before a release sign-off, an audit, or a decision about where to spend the next round of test effort.
中文简介正在准备中。当前显示原文说明。
Generates tests from natural-language specs (acceptance criteria, user stories, requirements) using an LLM, with confidence scoring per test case (LLM self-assessment plus heuristics: assertion quality, naming, completeness), batching uncertain cases for human review, and integration with the team's existing test framework. Use when the user asks to generate unit tests from acceptance criteria, convert user stories to test cases, automate test creation from requirements, or augment a spec-driven test suite with AI-generated stubs that are then curated before merge.
中文简介正在准备中。当前显示原文说明。
Configures Allure Report (test-runner adapter install, `allure-results` directory wiring, `categories.json` for failure classification, `history-trend.json` retention via the copy-history-between-runs pattern), runs the Allure CLI to convert `allure-results` to a static HTML site, and uploads the report as a CI artifact. Use when the team needs richer test reporting than JUnit XML - step-level attachments, per-test history, retry tracking, and severity / epic / feature labeling across framework-agnostic adapters (pytest, Jest, JUnit, TestNG, NUnit, Mocha). As a rich static HTML report generator, it is the open-source alternative to the sunset ExtentReports (JVM/.NET per-test HTML narrative); for hosted cross-run flakiness analytics rather than a static per-run report use currents-integration.
中文简介正在准备中。当前显示原文说明。