Yang Ruofeng

Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family D (experiment-forensics) cannot reach. Three patterns: train/test leakage means the reported score may not measure generalization (HP-EVAL-LEAKAGE — adopts the Kapoor & Narayanan 8-type / 3-category leakage taxonomy; the illegitimate-proxy / sampling-bias / pretraining-contamination subtypes hand off as needs_external_check, naming but NEVER running Oren-2023 exchangeability / Shi-2023 Min-K% / Golchin-2023 Time-Travel / BIG-bench canary); a load-bearing LLM judge is conflicted (same model/family as a compared system) or unvalidated (no human-agreement, no bias control) (HP-JUDGE-VALIDITY); a declared condition/metric is dropped or switched to favor the method, or 'best' is chosen with no held-out set (HP-SELECTIVE-REPORTING). Verdict-bearing at L0/L1 from the DESCRIBED protocol — NOT repo-gated like experiment-forensics; L2 only CONFIRMS against split/preprocessing/result files. A fresh cross-model reviewer (gpt-5.6-sol xhigh, read-only, fresh thread per pass) PROPOSES findings, each span-anchored to a ledger claim_id; tools/adjudicate_findings.py DECIDES the verdict. Leakage and under-reporting are usually HONEST methodological errors — every finding describes a discrepancy to CHECK, never an accusation. An LLM generating GROUND-TRUTH labels is HP-FAKE-GT (experiment-forensics) — routed there, not here. Emits eval-design-forensics.findings.json; computes NO verdict. Detect-only. Triggers: "eval design audit", "evaluation validity", "train/test leakage", "data leakage", "is the score measuring generalization", "LLM judge bias", "is the judge validated", "selective reporting", "cherry-picked results", "评估设计审计", "评测有效性", "数据泄漏", "训练测试集泄漏", "裁判模型有没有验证", "选择性报告".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

End-to-end substantive-integrity forensic sweep of a research paper (especially autoresearch / AI-Scientist-style output). Orchestrates the whole pipeline: ingest (arxiv-id | pdf | dir → working dir + pdftotext for L0) → /evidence-ledger (artifact manifest + observability level L0/L1/L2 + span-anchored claims.json) → fan out the integrity auditor skills (consistency, citation, baseline, experiment, presentation, proof-derivation, eval-design — each reads the ledger, emits span-anchored findings) + the zero-verdict-weight AIS writing-style track → advisory memos (/adversarial-case-builder + /novelty-duplication-advisory, no verdict weight) → deterministic tools/adjudicate_findings.py (--ledger REQUIRED) → reviewer-ready Integrity Forensics Report. Cross-model (fresh codex per dimension) and reviewer≠adjudicator: the model proposes findings, the deterministic adjudicator decides the verdict. Observability-aware, detect-only, never an opaque AI-text classifier (a separate zero-weight AIS section lists AI writing-style impressions, never moving the verdict). Triggers: "anti-autoresearch", "integrity audit this paper", "forensic review", "audit a submission", "审一篇投稿的诚信".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Synthesize the single strongest EVIDENCE-BOUND reviewer case to reject a paper, built ONLY from the evidence ledger (claims.json) + the other auditors' confirmed findings — never free-floating LLM critique. Two fresh cross-model codex threads: an attack writes the ~200-word rejection paragraph (every accusation tagged to an existing claim_id/finding_id), a defense decomposes it and rules each point against the anchored evidence. MEMO-ONLY: emits adversarial-case-builder.memo.md (fed to the adjudicator via --memo) and carries NO verdict weight — tools/adjudicate_findings.py lists it in ZERO_WEIGHT_SKILLS and caps it at info. Honest-null allowed (the paper may survive). Run LAST. Detect-only. Adapted from ARIS kill-argument. Triggers: "adversarial case", "strongest objection", "rejection memo", "kill argument", "最强拒稿点".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Citation-integrity forensics: is every reference real, correctly attributed, and used in a context the cited work actually supports? Catches hallucinated references (no paper at the claimed arXiv id/DOI/venue, fabricated authors/year), metadata drift (wrong year/venue/version), and wrong-context citations (a real paper cited for a claim it never makes — or argues against). A hot zone for machine-generated papers. Decidable at L0 (text + canonical sources). Span-anchored to the evidence ledger (claims.json); the executor gathers canonical facts (DBLP / arXiv / DOI), then one FRESH cross-model thread per cited key proposes findings; reviewer != adjudicator. Emits citation-forensics.findings.json; NEVER computes the verdict. Triggers: "citation forensics", "check the references", "hallucinated citations", "wrong-context citation", "verify references", "引用核对".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build the deterministic evidence ledger (artifact_manifest.json + claims.json) that every other Anti-Autoresearch auditor reads. One pass inventories artifacts, derives the observability level (L0 PDF-only / L1 +LaTeX / L2 +repo+results) by fixed rule, and extracts span-anchored, hashed, checkable claims (numbers, comparisons, scope, method, baselines, citations, captions, table cells) into claims.json. An OPTIONAL additive cross-model pass ADDS span-anchored semantic claims — method, theorem statements with their assumptions, definitions, proof/derivation steps and equations, scope, baselines, conclusions, the motivation span, and reproducibility-artifact references (the proof, derivation, and structure anchors the family B/D/G auditors need) — it never invents a number, emits a finding, or computes a verdict. Run FIRST, before any audit skill. Triggers: "build the ledger", "extract claims", "prep for integrity audit", "evidence ledger", "建证据账本".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Transparent, itemized impressions of AI-generated WRITING STYLE — the repo's ONLY non-integrity track. Two passes: a deterministic defensive-hedge density screen (tools/check_ai_style.py, AIS-DEFENSIVE-HEDGE) plus a fresh cross-model GROSS-cases-only semantic pass over the 13 AIS-* style tells (broken narrative arc, LLM phrase tics, jargon-stuffing, invented codenames, clause/formula walls, gratuitous pseudocode, bullet overuse, bold-module spam, restatement loops, focus drift, single-style figures, appendix dumping). Every finding is named, LOCATED, span-anchored to the evidence ledger (claims.json), carries not_integrity_finding:true + false_positive_risk:high + an fp_case, and gets ZERO verdict weight: the adjudicator forces it to info, excludes it from overall_verdict, and renders it in a SEPARATE report section. NOT an AI-text classifier — no scores, no "this is AI-written", no authorship probability; a paper can be CLEAN_GIVEN_EVIDENCE and still list many. Emits ai-style-impressions.findings.json; computes NO verdict; if a tell is actually substantive it routes to the integrity auditor. Triggers: "AI style", "vibe check", "writing fingerprint", "AI 文风", "vibe paper".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

MEMO-ONLY prior-work overlap advisory: surfaces the two ADVISORY taxonomy signals neither a tool nor a model can decide from the paper alone — ADV-TRIVIAL-COMBINATION (standard A+B+C / 缝合 stapling) and ADV-DUPLICATE-PUBLICATION (repackaged / duplicate submission). The executor RETRIEVES candidate prior work (DBLP fuzzy-title + boolean · WebSearch · WebFetch) from the paper's own title + contribution spans in the evidence ledger; TWO fresh cross-model codex reviewers (one per axis) LAY OUT the overlap side-by-side against each anchored contribution claim. It NEVER rules 'trivial' or 'duplicate' (that is a human judgment) and absence of a match is NOT evidence of originality. Emits novelty-duplication-advisory.memo.md + an info-only findings mirror; carries NO verdict weight — tools/adjudicate_findings.py lists it in ZERO_WEIGHT_SKILLS and caps it at info. Detect-only. Adapted from ARIS novelty-check, reframed from 'is MY idea novel' to 'here is the overlap a reviewer should weigh'. Triggers: "novelty advisory", "duplication check", "prior-work overlap", "is this stapling", "缝合", "查重", "重复发表", "duplicate submission".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Family-G proof & derivation integrity forensics: does a THIRD PARTY's written proof/derivation actually establish its theorem, or does it skip an obligation, assume its own conclusion, take an invalid step, drift a symbol's meaning, or smuggle an unstated assumption? Decides from the WRITTEN proof/derivation — verdict-bearing at L1 (the LaTeX source; PDF-extracted math is unreliable, so an L0 PDF-only run surfaces info only) — never asserts 'fabricated', only that the step shown does not hold. A fresh cross-model reviewer reads the theorem/proof + an extraction-only obligation scaffold and proposes per-obligation findings, each span-anchored to the evidence ledger (claims.json); reviewer≠adjudicator. Emits proof-derivation-forensics.findings.json; NEVER computes the verdict. dimension=proof, can be critical. Triggers: "proof forensics", "check this proof", "derivation integrity", "audit the math", "证明审计", "推导有没有漏洞".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Flagship intra-paper self-consistency forensics: does the paper contradict ITSELF across abstract/intro/tables/body/appendix, and does the method DESCRIBED match the method EVALUATED? Needs no external ground truth — works PDF-only (L0). Runs a deterministic arithmetic pass + a fresh cross-model semantic pass, every finding span-anchored to the evidence ledger (claims.json), reviewer≠adjudicator. Emits consistency-audit.findings.json; NEVER computes the verdict. Triggers: "consistency audit", "check the paper against itself", "self-consistency", "内部自洽".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Checkable-ish surface presentation signals a reviewer notices first — duplicate/near-identical tables, leftover pipeline/template strings, too-few or LLM-looking figures, and page-padding. AUXILIARY ONLY and weak by design: a deterministic pass (tools/check_presentation.py — dup-table + pipeline-artifact) plus a fresh cross-model GROSS-cases-only semantic pass (thin-float, LLM-figure, page-padding), every above-info finding span-anchored to the evidence ledger (claims.json). The adjudicator CAPS everything at minor (SURFACE_ONLY_SKILLS + SURFACE_PATTERNS) — these contribute at most SOFT_FLAGS, never a HARD verdict — default false_positive_risk:high. NOTE: the pure AI writing-STYLE impressions (AI-flavor prose, defensive 'not-X-but-Y' hedging, narrative-arc, jargon-stuffing, invented codenames) MOVED to the zero-verdict-weight AIS track — for those use skills/ai-style-impressions, NOT this. Emits presentation-signals.findings.json; NEVER computes the verdict. Triggers: "presentation signals", "surface check", "duplicate tables", "排版信号".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN-video for Claude Code with independent Codex adversarial review. Create or revise a video through agent-authored code, rendering, editing and fresh-context audiovisual review, without image, video or music generation models. Use for reference-led animation, procedural films, motion design, explainers, construction replays, and continuing an existing coded-video project. Carries the work from reference study and animatic through a finished, reproducible film; supports explicitly requested self-review when appropriate.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN — Agentic Loop Image GeneratioN. Paint a subject in the craft of a reference artwork as executable p5.js, Canvas, SVG or another mark-based medium, replay that craft's real order of work, and improve it through pixel-only cross-model review. The reference fixes the craft; the subject may be the reference's own scene (千里江山图 → p5.js) or another (清明上河图 → 武汉). Not for one-shot raster image generation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN — Agentic Loop Image GeneratioN. Paint a subject in the craft of a reference artwork as executable p5.js, Canvas, SVG or another mark-based medium, replay that craft's real order of work, and improve it through pixel-only independent Codex subagent review. The reference fixes the craft; the subject may be the reference's own scene (千里江山图 → p5.js) or another (清明上河图 → 武汉). Not for one-shot raster image generation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN for paper figures — draw a method-overview figure as an executable p5.js program that replays its own construction in named stages, exports SVG, and improves through pixel-only cross-model review whose gate is 'explain the method back from the figure alone, and is it worth looking at'. The method sets the relationships the figure must show, the chosen visual identity sets how its marks look, a composition source lends arrangements; the method may be the source's own (意临) or yours in its arrangement (风格迁移). Not for one-shot raster image generation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN-video for Codex. Create or revise a video through agent-authored code, rendering, editing and fresh-context audiovisual review, without image, video or music generation models. Use for reference-led animation, procedural films, motion design, explainers, construction replays, and continuing an existing coded-video project. Carries the work from reference study and animatic through a finished, reproducible film; supports explicitly requested self-review when appropriate.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

ALIGN for paper figures — draw a method-overview figure as an executable p5.js program that replays its own construction in named stages, exports SVG, and improves through pixel-only independent Codex subagent review whose gate is 'explain the method back from the figure alone, and is it worth looking at'. The method sets the relationships the figure must show, the chosen visual identity sets how its marks look, a composition source lends arrangements; the method may be the source's own (意临) or yours in its arrangement (风格迁移). Not for one-shot raster image generation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.