Skip to content
cccskills
FreePublished on GitHub

auditing-data-and-ground-truth

Audit datasets, joins, labels, and ground truth before analysis or modeling. Use when data meaning, row grain, time semantics, source-of-truth reliability, or label construction may invalidate conclusions; not for general model evaluation after the evidence base is already trusted.

The English summary is being prepared. Showing the original description.

See how to install

Files included (4)

  • SKILL.md3.7 KB
  • agents/openai.yaml250 B
  • assets/data-and-ground-truth-audit.md2.5 KB
  • references/audit-playbook.md4.0 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Audit Data and Ground Truth

Determine what the data actually represents and whether its labels can support the intended claim. The outcome is an evidence-backed readiness judgment, not a generic profile report.

Anchor the audit

State the intended decision, entity, row grain, target, prediction moment, outcome window, and population. If these are not known, surface that uncertainty before interpreting columns or labels.

Preserve source data unchanged. Work from read-only inputs or documented snapshots, and keep every correction, exclusion, or derived label traceable to its source.

Inspect in risk order

  1. Source and lineage: Identify origin, owner, extraction time, filters, transformations, versions, and competing sources of truth.
  2. Structure: Measure row and entity counts, schema, types, key uniqueness, duplicates, missingness, ranges, and category cardinality. Report both counts and rates where scale matters.
  3. Meaning: Confirm units, code sets, null semantics, sentinel values, event meanings, and whether one row means what the analysis assumes.
  4. Relationships: Test key uniqueness on each side before joining. State expected and observed cardinality, unmatched rates, row multiplication, and aggregation effects.
  5. Time: Separate event time, record time, update time, and availability time. Check impossible orderings, late arrivals, backfills, and future knowledge.
  6. Ground truth: Trace each label from operational event to encoded value. Verify annotator or system provenance, adjudication, label maturity, class definitions, coverage, disagreement, and missing-label behavior.
  7. Reconciliation: Compare raw records with independent records or a stratified sample. Do not let agreement with a downstream table validate the upstream process that created both.

Read references/audit-playbook.md when selecting concrete tests or grading the severity of a finding.

Protect semantics

  • Do not silently interpret missing as negative, absence as zero, or the latest value as historically available.
  • Do not deduplicate until the real-world uniqueness rule is established.
  • Do not repair values in place; produce a derived correction with a reason and impact count.
  • Do not accept labels merely because they are named target, truth, gold, or final.
  • Do not average away disagreements that reveal a definition or annotation problem.

Within already authorized data, systems, and compute, run read-only or reversible tests and document the evidence. Obtain approval before choosing a new source of truth, changing label policy, excluding a material population, applying irreversible corrections, accessing a new or sensitive source, incurring material cost, involving an external reviewer, or sharing records outside the approved boundary.

Issue a readiness judgment

Use assets/data-and-ground-truth-audit.md for the deliverable. For every material finding, record evidence, affected scope, consequence, and recommended treatment. End with one status:

  • ready: evidence supports the intended use;
  • ready with conditions: use is supportable only under named constraints;
  • not ready: a known defect can change the conclusion;
  • unknown: critical evidence is missing or cannot be independently checked.

Never upgrade unknown to ready because no defect was observed.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Design leakage-safe machine learning experiments that mirror real deployment and support fair model comparisons. Use when defining prediction timing, feature eligibility, train-validation-test splits, baselines, metrics, or controlled model iterations; not for auditing whether raw labels are trustworthy.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

Isolate the root cause of ML performance drops, inconsistent evaluations, prediction errors, and training-serving mismatches across data, labels, splits, pipelines, models, metrics, and runtime behavior. Use when investigating a reproducible failure or regression, not routine model selection or general performance validation.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

Orchestrate an end-to-end data analysis or machine learning project from decision framing through reproducible handoff. Use when a request spans multiple lifecycle stages or an ambiguous modeling request must become a decision-ready result; use narrower audit or experiment-design skills for isolated reviews.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

Package completed data analysis and ML work so an independent recipient can reproduce the claimed results, verify artifact lineage, and operate the handoff within its stated scope. Use when finalizing a project, study, model package, or review bundle; not for deploying to a live system.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

Route data analysis and machine learning work to the right skill in this suite. Use when starting any analysis, modeling, validation, or reproducibility task and the matching specialized skill is not yet clear; not needed when one specific skill already clearly applies.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

Validate trained models and analytical claims against their intended decision, independent evidence, and human-reviewed ground truth. Use when reviewing model performance, analysis conclusions, launch claims, or evaluation reports; use failure diagnosis instead when the main task is locating a known defect.

The English summary is being prepared. Showing the original description.

aiopshwang/data-analysis-ml-agent-skills13Updated Aug 27, 2026

See all skills by aiopshwang

Report a problem with this skill