Citation Integrity Auditor

Checks that every cited work exists, that its identifier points to that work, that the text and the reference list agree, and that each source can bear the claim attached to it. Verification comes from bibliographic indexes, not from model memory: a lookup that was not run is reported as NOT CHECKED, never filled in by plausibility.

Adapted for social science from the source-verification workflow in Cheng-I Wu's Academic Research Skills for Claude Code (CC BY-NC 4.0), in particular its separation of formatting, existence, DOI, and uncertainty checks. Credit ARS when reusing this structure elsewhere, and keep its prose out of reports.

Instructions

1. Orient before checking

Identify the inputs and constraints:

• Manuscript source: LaTeX, Markdown, Word export, PDF text, or pasted prose.
• Bibliography source: .bib, CSL JSON, Zotero export, reference section, footnotes, or mixed.
• Target style: default to APA 7 unless the user names a journal or style.
• Scope: whole manuscript, one section, bibliography only, or DOI-only pass.
• Tool availability: if web lookup, Crossref, DataCite, DOI.org, Semantic Scholar, or OpenAlex are unavailable, mark verification status as NOT CHECKED rather than guessing.
• For LaTeX/BibTeX, drive the audit off the keys actually \cited in the manuscript (follow \input/\include), not every entry in a master .bib. Only cited keys reach the printed reference list, which is what readers and critics actually check.
• Note the bibliography style. Many common styles (apalike, plain, unsrt) do not render the doi field, so a wrong or dead DOI is invisible in the compiled PDF and ranks as source-hygiene unless the style prints it. Author, year, title, volume, issue, and pages do render — prioritize those for any artifact that is already public.

2. Build the citation inventory

Create two inventories before judging errors:

• In-text inventory: every author-year, parenthetical, narrative, footnote, or numeric citation appearing in the manuscript.
• Reference inventory: every bibliography entry, including entry key if present, authors, year, title, outlet, DOI/URL, publication status, and notes.

Normalize cautiously:

• Treat spelling variants, diacritics, hyphenated surnames, particles (de, van, von), and organizational authors as possible matches until checked.
• For multiple works by the same author-year, verify a/b/c suffixes are assigned consistently.
• For BibTeX, preserve the entry key and report it with every finding.

3. Run structural checks

Report:

• Cited in text but missing from references.
• In references but never cited.
• Author-year mismatch between in-text citation and reference entry.
• Duplicate bibliography entries for the same work.
• Ambiguous citations where two reference entries could satisfy one in-text citation.
• Broken cross-references caused by LaTeX/BibTeX key drift.

List the likely match and the evidence for ambiguous cases rather than resolving them silently.

4. Verify source existence and identifiers

This is the highest-stakes check. The two common fabrication patterns are a real author attached to a plausible but nonexistent title, and a DOI that resolves to a different real paper. Query the programmatic indexes first: their answers are deterministic and do not depend on what the model remembers.

Concrete lookups (via web fetch):

• Crossref bibliographic search: https://api.crossref.org/works?rows=5&query.bibliographic=<URL-encoded "title first-author year">. Compare returned title/authors/year/DOI. Append &mailto=<email> for the polite pool. On HTTP 429, fall back to OpenAlex.
• DOI resolution: https://api.crossref.org/works/<doi> (or https://doi.org/<doi>). Confirm the resolved title and authors match this entry. A DOI that resolves to a different real work is the most common LLM-fabrication signature, so a DOI that resolves is not thereby correct.
• OpenAlex: https://api.openalex.org/works?search=<URL-encoded title> or ?filter=doi:<doi>. Independent index; use when Crossref is rate-limited or for abstracts.
• Exact-title test: search the title in quotation marks. Zero hits anywhere is a strong fabrication signal.
• Author-corpus cross-check (catches the "fabricated title grafted onto a real author" pattern): query the real author's works (...query.author=<name>) and confirm the cited title appears in their corpus. A real author + nonexistent title + invented co-authors/venue is the classic hallucination.

Tiered fallback by source type:

• Journal articles → Crossref / OpenAlex / DOI resolution.
• Books and chapters → publisher page, WorldCat, Google Books, OpenAlex. Book DOIs are spotty and frequently 404 even for real books; a dead book DOI is usually source-hygiene, not a missing source.
• Datasets and official statistics → the issuing archive (ISSP, V-Dem, ANES, KGSS, national election studies, government statistical catalogues). Verify the catalogue/series number resolves to the cited title — a real catalogue number attached to the wrong title is a fabrication tell.
• Grey literature / government / internal reports → the issuing agency's own site (site: search). If untraceable, classify as NEEDS AUTHOR VERIFICATION, not fabricated — an insider or co-author may be citing a real internal document.
• News / web items → confirm the named outlet actually ran the piece.
• Preprints / "forthcoming" → SSRN, arXiv, SocArXiv, OSF, author pages/CVs, conference programs. A forthcoming item's year is inherently uncertain; prefer the authors' own current listing.

Labels:

• MISSING DOI: likely DOI exists but is absent.
• NO DOI FOUND: searched, none found; not necessarily an error.
• DEAD DOI: the identifier 404s (common for real book DOIs).
• DOI RESOLVES TO DIFFERENT WORK: DOI is live but points to another paper — high-signal hallucination tell.
• METADATA MISMATCH: real work, wrong year/volume/issue/pages/author names.
• TITLE DRIFT: cited title differs materially from the indexed title.
• STATUS UPDATE: working paper/preprint now has a journal/book version.
• NEEDS AUTHOR VERIFICATION: no programmatic match, but plausibly real grey literature.
• LIKELY FABRICATED: no trace after identifier, exact-title, and author-corpus searches. Reserve for genuine non-existence, but do not shy away from it when the evidence is clear.

Not fabrications:

• Double-blind anonymization placeholders (e.g. title={Article withheld for review}, author {Author}) — intentional blinding stubs, not citations.
• The author's own work (self-citations). Flag metadata conflicts between their CV and the published record for the author to reconcile; the author is the authority on their own paper.

High-stakes audits (publication or public posting): run a second, independent verification pass with different framing, treating any suspect as fabricated until independently confirmed. A single pass misses real problems, and independent passes catch each other's gaps. If no second pass was run, say so in the report. When remediating rather than only reporting, substitute only a real, verified source (ideally the one the author intended), confirm any new DOI resolves to the right work before writing it, and check that the fix renders in the compiled output.

5. Check style and completeness

For APA 7 and most social-science journal styles:

• DOI should be a URL form when available: https://doi.org/....
• Do not add a period after DOI or URL.
• Journal articles need article title, journal title, volume, issue when available, pages or article number, and DOI when available.
• Books need publisher; edited chapters need editors, book title, page range, publisher.
• Reports and datasets need institutional author, date, title, publisher/archive, version if relevant, and URL/DOI.
• Data, code, and materials should be cited as research products, not buried only in availability statements.

When target style is not APA, apply the target style but still keep the integrity checks above.

6. Audit evidentiary use

For literature reviews and theory sections, sample the most important citations and classify how they are used:

• Background: source establishes context only.
• Claim support: source is used as evidence for a substantive claim.
• Method precedent: source justifies a design, estimator, measurement strategy, or reporting practice.
• Contrast: source is cited as an opposing or incomplete account.

Flag overclaiming when the cited source plausibly cannot support the sentence where it appears. If source text is unavailable, mark SOURCE TEXT NOT CHECKED.

Output

Produce a Citation Audit Report:

# Citation Audit Report

Scope: <files/sections checked>
Target style: <style or journal>
Verification tools used: <Crossref/DataCite/S2/OpenAlex/Web/manual/none>
Summary: <N blocking, N recommended, N minor, N needs-author-verification>

## Blocking Issues
| Location | Entry/key | Issue | Evidence | Fix |

## Recommended Fixes
| Location | Entry/key | Issue | Evidence | Fix |

## Minor Style Issues
| Location | Entry/key | Issue | Fix |

## DOI and Source-Status Table
| Entry/key | Current DOI/URL | Verification status | Suggested action |

## In-text / Reference Parity
| Type | Citation or key | Location | Suggested action |

## Needs Author Verification
| Entry/key | Why unresolved | What author should check |

Severity:

• Blocking: missing cited reference, wrong DOI, possibly nonexistent source, citation supports a decisive claim but appears wrong, or broken citation build.
• Recommended: missing likely DOI, stale working paper, author-year mismatch that is fixable, uncited important reference, incomplete metadata.
• Minor: punctuation, capitalization, inconsistent initials, style-only issues.

Optional Jev verification

$citation-check --verify <task> opts into experimental Jev verification; $citation-check --verify jev <task> is identical. Plain $citation-check is unchanged. It may assist only with an ambiguity left after actual, fetched metadata: work identity, version relationship, or otherwise ambiguous metadata. It never replaces the inventory, DOI resolution, existence checks, provider lookups, style audit, or a lead's report label.

Use it only after the ordinary citation-check workflow has retrieved the relevant record. A lead may manually refer to a prior result only for the same complete evaluation input and model/contract version; this is not automatic caching. A deterministic wrong DOI remains DOI RESOLVES TO DIFFERENT WORK; NOT CHECKED remains NOT CHECKED. A DOI difference alone can be a preprint/publication or other version relationship, not proof of a wrong work. Only on opt-in, read ${JEV_VERIFIER_HOME:-$HOME/.local/share/oss-experiments/jev-verifier/current}/CITATION-CHECK.md.