スキルを探す

Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family D (experiment-forensics) cannot reach. Three patterns: train/test leakage means the reported score may not measure generalization (HP-EVAL-LEAKAGE — adopts the Kapoor & Narayanan 8-type / 3-category leakage taxonomy; the illegitimate-proxy / sampling-bias / pretraining-contamination subtypes hand off as needs_external_check, naming but NEVER running Oren-2023 exchangeability / Shi-2023 Min-K% / Golchin-2023 Time-Travel / BIG-bench canary); a load-bearing LLM judge is conflicted (same model/family as a compared system) or unvalidated (no human-agreement, no bias control) (HP-JUDGE-VALIDITY); a declared condition/metric is dropped or switched to favor the method, or 'best' is chosen with no held-out set (HP-SELECTIVE-REPORTING). Verdict-bearing at L0/L1 from the DESCRIBED protocol — NOT repo-gated like experiment-forensics; L2 only CONFIRMS against split/preprocessing/result files. A fresh cross-model reviewer (gpt-5.6-sol xhigh, read-only, fresh thread per pass) PROPOSES findings, each span-anchored to a ledger claim_id; tools/adjudicate_findings.py DECIDES the verdict. Leakage and under-reporting are usually HONEST methodological errors — every finding describes a discrepancy to CHECK, never an accusation. An LLM generating GROUND-TRUTH labels is HP-FAKE-GT (experiment-forensics) — routed there, not here. Emits eval-design-forensics.findings.json; computes NO verdict. Detect-only. Triggers: "eval design audit", "evaluation validity", "train/test leakage", "data leakage", "is the score measuring generalization", "LLM judge bias", "is the judge validated", "selective reporting", "cherry-picked results", "评估设计审计", "评测有效性", "数据泄漏", "训练测试集泄漏", "裁判模型有没有验证", "选择性报告".

日本語の概要は準備中です。原文の説明を表示しています。

Build the deterministic evidence ledger (artifact_manifest.json + claims.json) that every other Anti-Autoresearch auditor reads. One pass inventories artifacts, derives the observability level (L0 PDF-only / L1 +LaTeX / L2 +repo+results) by fixed rule, and extracts span-anchored, hashed, checkable claims (numbers, comparisons, scope, method, baselines, citations, captions, table cells) into claims.json. An OPTIONAL additive cross-model pass ADDS span-anchored semantic claims — method, theorem statements with their assumptions, definitions, proof/derivation steps and equations, scope, baselines, conclusions, the motivation span, and reproducibility-artifact references (the proof, derivation, and structure anchors the family B/D/G auditors need) — it never invents a number, emits a finding, or computes a verdict. Run FIRST, before any audit skill. Triggers: "build the ledger", "extract claims", "prep for integrity audit", "evidence ledger", "建证据账本".

日本語の概要は準備中です。原文の説明を表示しています。

Checkable-ish surface presentation signals a reviewer notices first — duplicate/near-identical tables, leftover pipeline/template strings, too-few or LLM-looking figures, and page-padding. AUXILIARY ONLY and weak by design: a deterministic pass (tools/check_presentation.py — dup-table + pipeline-artifact) plus a fresh cross-model GROSS-cases-only semantic pass (thin-float, LLM-figure, page-padding), every above-info finding span-anchored to the evidence ledger (claims.json). The adjudicator CAPS everything at minor (SURFACE_ONLY_SKILLS + SURFACE_PATTERNS) — these contribute at most SOFT_FLAGS, never a HARD verdict — default false_positive_risk:high. NOTE: the pure AI writing-STYLE impressions (AI-flavor prose, defensive 'not-X-but-Y' hedging, narrative-arc, jargon-stuffing, invented codenames) MOVED to the zero-verdict-weight AIS track — for those use skills/ai-style-impressions, NOT this. Emits presentation-signals.findings.json; NEVER computes the verdict. Triggers: "presentation signals", "surface check", "duplicate tables", "排版信号".

日本語の概要は準備中です。原文の説明を表示しています。

Citation-integrity forensics: is every reference real, correctly attributed, and used in a context the cited work actually supports? Catches hallucinated references (no paper at the claimed arXiv id/DOI/venue, fabricated authors/year), metadata drift (wrong year/venue/version), and wrong-context citations (a real paper cited for a claim it never makes — or argues against). A hot zone for machine-generated papers. Decidable at L0 (text + canonical sources). Span-anchored to the evidence ledger (claims.json); the executor gathers canonical facts (DBLP / arXiv / DOI), then one FRESH cross-model thread per cited key proposes findings; reviewer != adjudicator. Emits citation-forensics.findings.json; NEVER computes the verdict. Triggers: "citation forensics", "check the references", "hallucinated citations", "wrong-context citation", "verify references", "引用核对".

日本語の概要は準備中です。原文の説明を表示しています。

End-to-end substantive-integrity forensic sweep of a research paper (especially autoresearch / AI-Scientist-style output). Orchestrates the whole pipeline: ingest (arxiv-id | pdf | dir → working dir + pdftotext for L0) → /evidence-ledger (artifact manifest + observability level L0/L1/L2 + span-anchored claims.json) → fan out the integrity auditor skills (consistency, citation, baseline, experiment, presentation, proof-derivation, eval-design — each reads the ledger, emits span-anchored findings) + the zero-verdict-weight AIS writing-style track → advisory memos (/adversarial-case-builder + /novelty-duplication-advisory, no verdict weight) → deterministic tools/adjudicate_findings.py (--ledger REQUIRED) → reviewer-ready Integrity Forensics Report. Cross-model (fresh codex per dimension) and reviewer≠adjudicator: the model proposes findings, the deterministic adjudicator decides the verdict. Observability-aware, detect-only, never an opaque AI-text classifier (a separate zero-weight AIS section lists AI writing-style impressions, never moving the verdict). Triggers: "anti-autoresearch", "integrity audit this paper", "forensic review", "audit a submission", "审一篇投稿的诚信".

日本語の概要は準備中です。原文の説明を表示しています。

Audit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/configs its results need) — every finding span-anchored to a ledger claim_id. At L0/L1 (PDF / source only) the same patterns are surfaced as info-level 'could-not-verify' signals where the ledger gives an anchor (observability_level_required:2) — NEVER a fraud verdict from a PDF. The reviewer PROPOSES findings; tools/adjudicate_findings.py computes the verdict. Detect-only. Triggers: "experiment forensics", "audit the results", "check the eval code", "实验诚实度".

日本語の概要は準備中です。原文の説明を表示しています。

MEMO-ONLY prior-work overlap advisory: surfaces the two ADVISORY taxonomy signals neither a tool nor a model can decide from the paper alone — ADV-TRIVIAL-COMBINATION (standard A+B+C / 缝合 stapling) and ADV-DUPLICATE-PUBLICATION (repackaged / duplicate submission). The executor RETRIEVES candidate prior work (DBLP fuzzy-title + boolean · WebSearch · WebFetch) from the paper's own title + contribution spans in the evidence ledger; TWO fresh cross-model codex reviewers (one per axis) LAY OUT the overlap side-by-side against each anchored contribution claim. It NEVER rules 'trivial' or 'duplicate' (that is a human judgment) and absence of a match is NOT evidence of originality. Emits novelty-duplication-advisory.memo.md + an info-only findings mirror; carries NO verdict weight — tools/adjudicate_findings.py lists it in ZERO_WEIGHT_SKILLS and caps it at info. Detect-only. Adapted from ARIS novelty-check, reframed from 'is MY idea novel' to 'here is the overlap a reviewer should weigh'. Triggers: "novelty advisory", "duplication check", "prior-work overlap", "is this stapling", "缝合", "查重", "重复发表", "duplicate submission".

日本語の概要は準備中です。原文の説明を表示しています。

Transparent, itemized impressions of AI-generated WRITING STYLE — the repo's ONLY non-integrity track. Two passes: a deterministic defensive-hedge density screen (tools/check_ai_style.py, AIS-DEFENSIVE-HEDGE) plus a fresh cross-model GROSS-cases-only semantic pass over the 13 AIS-* style tells (broken narrative arc, LLM phrase tics, jargon-stuffing, invented codenames, clause/formula walls, gratuitous pseudocode, bullet overuse, bold-module spam, restatement loops, focus drift, single-style figures, appendix dumping). Every finding is named, LOCATED, span-anchored to the evidence ledger (claims.json), carries not_integrity_finding:true + false_positive_risk:high + an fp_case, and gets ZERO verdict weight: the adjudicator forces it to info, excludes it from overall_verdict, and renders it in a SEPARATE report section. NOT an AI-text classifier — no scores, no "this is AI-written", no authorship probability; a paper can be CLEAN_GIVEN_EVIDENCE and still list many. Emits ai-style-impressions.findings.json; computes NO verdict; if a tell is actually substantive it routes to the integrity auditor. Triggers: "AI style", "vibe check", "writing fingerprint", "AI 文风", "vibe paper".

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN — Agentic Loop Image GeneratioN. Paint a subject in the craft of a reference artwork as executable p5.js, Canvas, SVG or another mark-based medium, replay that craft's real order of work, and improve it through pixel-only independent Codex subagent review. The reference fixes the craft; the subject may be the reference's own scene (千里江山图 → p5.js) or another (清明上河图 → 武汉). Not for one-shot raster image generation.

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN for paper figures — draw a method-overview figure as an executable p5.js program that replays its own construction in named stages, exports SVG, and improves through pixel-only cross-model review whose gate is 'explain the method back from the figure alone, and is it worth looking at'. The method sets the relationships the figure must show, the chosen visual identity sets how its marks look, a composition source lends arrangements; the method may be the source's own (意临) or yours in its arrangement (风格迁移). Not for one-shot raster image generation.

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN-video for Claude Code with independent Codex adversarial review. Create or revise a video through agent-authored code, rendering, editing and fresh-context audiovisual review, without image, video or music generation models. Use for reference-led animation, procedural films, motion design, explainers, construction replays, and continuing an existing coded-video project. Carries the work from reference study and animatic through a finished, reproducible film; supports explicitly requested self-review when appropriate.

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN — Agentic Loop Image GeneratioN. Paint a subject in the craft of a reference artwork as executable p5.js, Canvas, SVG or another mark-based medium, replay that craft's real order of work, and improve it through pixel-only cross-model review. The reference fixes the craft; the subject may be the reference's own scene (千里江山图 → p5.js) or another (清明上河图 → 武汉). Not for one-shot raster image generation.

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN-video for Codex. Create or revise a video through agent-authored code, rendering, editing and fresh-context audiovisual review, without image, video or music generation models. Use for reference-led animation, procedural films, motion design, explainers, construction replays, and continuing an existing coded-video project. Carries the work from reference study and animatic through a finished, reproducible film; supports explicitly requested self-review when appropriate.

日本語の概要は準備中です。原文の説明を表示しています。

ALIGN for paper figures — draw a method-overview figure as an executable p5.js program that replays its own construction in named stages, exports SVG, and improves through pixel-only independent Codex subagent review whose gate is 'explain the method back from the figure alone, and is it worth looking at'. The method sets the relationships the figure must show, the chosen visual identity sets how its marks look, a composition source lends arrangements; the method may be the source's own (意临) or yours in its arrangement (风格迁移). Not for one-shot raster image generation.

日本語の概要は準備中です。原文の説明を表示しています。