本文へ移動
cccskills
無料GitHub で公開

auditing-data-and-ground-truth

Audit datasets, joins, labels, and ground truth before analysis or modeling. Use when data meaning, row grain, time semantics, source-of-truth reliability, or label construction may invalidate conclusions; not for general model evaluation after the evidence base is already trusted.

インストール方法を見る

含まれるファイル(4)

  • SKILL.md3.7 KB
  • agents/openai.yaml250 B
  • assets/data-and-ground-truth-audit.md2.5 KB
  • references/audit-playbook.md4.0 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Audit Data and Ground Truth

Determine what the data actually represents and whether its labels can support the intended claim. The outcome is an evidence-backed readiness judgment, not a generic profile report.

Anchor the audit

State the intended decision, entity, row grain, target, prediction moment, outcome window, and population. If these are not known, surface that uncertainty before interpreting columns or labels.

Preserve source data unchanged. Work from read-only inputs or documented snapshots, and keep every correction, exclusion, or derived label traceable to its source.

Inspect in risk order

  1. Source and lineage: Identify origin, owner, extraction time, filters, transformations, versions, and competing sources of truth.
  2. Structure: Measure row and entity counts, schema, types, key uniqueness, duplicates, missingness, ranges, and category cardinality. Report both counts and rates where scale matters.
  3. Meaning: Confirm units, code sets, null semantics, sentinel values, event meanings, and whether one row means what the analysis assumes.
  4. Relationships: Test key uniqueness on each side before joining. State expected and observed cardinality, unmatched rates, row multiplication, and aggregation effects.
  5. Time: Separate event time, record time, update time, and availability time. Check impossible orderings, late arrivals, backfills, and future knowledge.
  6. Ground truth: Trace each label from operational event to encoded value. Verify annotator or system provenance, adjudication, label maturity, class definitions, coverage, disagreement, and missing-label behavior.
  7. Reconciliation: Compare raw records with independent records or a stratified sample. Do not let agreement with a downstream table validate the upstream process that created both.

Read references/audit-playbook.md when selecting concrete tests or grading the severity of a finding.

Protect semantics

  • Do not silently interpret missing as negative, absence as zero, or the latest value as historically available.
  • Do not deduplicate until the real-world uniqueness rule is established.
  • Do not repair values in place; produce a derived correction with a reason and impact count.
  • Do not accept labels merely because they are named target, truth, gold, or final.
  • Do not average away disagreements that reveal a definition or annotation problem.

Within already authorized data, systems, and compute, run read-only or reversible tests and document the evidence. Obtain approval before choosing a new source of truth, changing label policy, excluding a material population, applying irreversible corrections, accessing a new or sensitive source, incurring material cost, involving an external reviewer, or sharing records outside the approved boundary.

Issue a readiness judgment

Use assets/data-and-ground-truth-audit.md for the deliverable. For every material finding, record evidence, affected scope, consequence, and recommended treatment. End with one status:

  • ready: evidence supports the intended use;
  • ready with conditions: use is supportable only under named constraints;
  • not ready: a known defect can change the conclusion;
  • unknown: critical evidence is missing or cannot be independently checked.

Never upgrade unknown to ready because no defect was observed.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

Design leakage-safe machine learning experiments that mirror real deployment and support fair model comparisons. Use when defining prediction timing, feature eligibility, train-validation-test splits, baselines, metrics, or controlled model iterations; not for auditing whether raw labels are trustworthy.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

Isolate the root cause of ML performance drops, inconsistent evaluations, prediction errors, and training-serving mismatches across data, labels, splits, pipelines, models, metrics, and runtime behavior. Use when investigating a reproducible failure or regression, not routine model selection or general performance validation.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

Orchestrate an end-to-end data analysis or machine learning project from decision framing through reproducible handoff. Use when a request spans multiple lifecycle stages or an ambiguous modeling request must become a decision-ready result; use narrower audit or experiment-design skills for isolated reviews.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

Package completed data analysis and ML work so an independent recipient can reproduce the claimed results, verify artifact lineage, and operate the handoff within its stated scope. Use when finalizing a project, study, model package, or review bundle; not for deploying to a live system.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

Route data analysis and machine learning work to the right skill in this suite. Use when starting any analysis, modeling, validation, or reproducibility task and the matching specialized skill is not yet clear; not needed when one specific skill already clearly applies.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

Validate trained models and analytical claims against their intended decision, independent evidence, and human-reviewed ground truth. Use when reviewing model performance, analysis conclusions, launch claims, or evaluation reports; use failure diagnosis instead when the main task is locating a known defect.

日本語の概要は準備中です。原文の説明を表示しています。

aiopshwang/data-analysis-ml-agent-skills122026年8月27日 更新

aiopshwang のスキルをすべて見る

このスキルの問題を報告する