Documentation Sync Checker Skill

Deterministic 4-phase drift detector that compares the filesystem against README entries. Each phase (Scan, Cross-Reference, Detect, Report) has a gate that must pass before proceeding. The skill produces a sync score (percentage of tools properly documented) and actionable fix suggestions for every detected issue.

This skill checks documentation presence and absence only -- it does not judge description quality, generate documentation content, resolve merge conflicts, validate cross-references, or track when drift occurred. Suggested fixes use YAML descriptions verbatim; content generation and quality assessment require different skills.

Optional flags: --auto-fix (experimental, requires explicit opt-in), --strict (exit code 1 on issues), --format json (machine-readable output for CI/CD).

---

Instructions

Phase 1: SCAN

Goal: Discover all skills, agents, and commands in the repository filesystem. All discovery (file existence checks, YAML parsing, markdown extraction) must be deterministic -- no AI judgment on content quality.

Step 1: Run the scan script

python3 skills/meta/docs-sync-checker/scripts/scan_tools.py --repo-root $HOME/vexjoy-agent

Step 2: Validate discovery results

For each tool type, verify:

Skills (skills/**/SKILL.md):

• File has opening --- and closing --- YAML delimiters
• YAML contains non-empty name and description fields
• name field matches directory name (e.g., skills/code-quality/code-linting/ has name: code-linting)

Agents (agents/*.md):

• File has valid YAML frontmatter with name field
• Filename (without .md) matches YAML name value

Commands (commands/**/*.md):

• File exists as markdown in commands/ directory
• Namespaced commands in subdirectories (e.g., commands/code/cleanup.md) are detected

Step 3: Validate the docs routing catalog

Every docs/*.md file (outside archive/ and images/) carries frontmatter with summary and read_when — the on-demand load triggers for docs, matching what skills/INDEX.json gives skills.

python3 scripts/docs-catalog.py --check

Exit 1 means a doc is missing frontmatter; add summary and read_when to that file. python3 scripts/docs-catalog.py (no flags) prints the catalog table; --json emits it machine-readable.

Step 4: Count and verify

## Scan Results
Skills found: [N]
Agents found: [N]
Commands found: [N]
YAML errors: [N] (must be 0 to proceed)

Gate: All tools discovered, all YAML valid, counts >0 for each type, docs catalog check exits 0. Proceed only after the gate passes.

Phase 2: CROSS-REFERENCE

Goal: Extract documented tools from README files and compare with discovered tools. Each tool type has a primary documentation file: skills belong in docs/skills.md, agents in agents/README.md, commands in commands/README.md.

Step 1: Run the documentation parser

python3 skills/meta/docs-sync-checker/scripts/parse_docs.py --repo-root $HOME/vexjoy-agent --scan-results /tmp/scan_results.json

Step 2: Parse each documentation file

These are the five documentation files to check -- no others:

File	Format	What to Extract
docs/skills.md	Markdown table	Name, Description, Command, Hook columns
agents/README.md	Table or list	Name, Description fields
commands/README.md	Markdown list	/command-name - Description items
README.md	Inline references	Pattern-match skill: X, /command, agent-name
docs/REFERENCE.md	Section headers	### tool-name headers with descriptions

Step 3: Build documented-tools registry

For each documentation file, collect the set of tool names found. This creates a mapping of {file -> [tool_names]} that Phase 3 will compare against the filesystem scan.

Step 4: Verify parse completeness

• All 5 documentation files found and parsed (warn if any missing)
• No parse errors on table/list structures
• Tool names extracted from each file

Gate: All documentation files parsed without errors. Proceed only after the gate passes.

Phase 3: DETECT

Goal: Compare discovered tools with documented tools to identify drift. This is a point-in-time snapshot -- it cannot tell you when drift occurred, only that it exists now.

Step 1: Compute set differences

For each tool type and its primary documentation file:

• missing = filesystem_tools - documented_tools (tools that exist but are not documented)
• stale = documented_tools - filesystem_tools (documented tools that no longer exist -- users waste time trying to invoke non-existent tools, so always flag these)

Step 2: Categorize and assign severity

Severity reflects user impact: missing entries mean tools are undiscoverable and stale entries waste time.

Category	Condition	Severity
Missing Entry	Tool in filesystem, not in primary README	HIGH
Stale Entry	Tool in README, not in filesystem	MEDIUM
Incomplete Entry	Documentation missing required fields	LOW

Step 3: Record issue details

For each issue, capture: tool type, tool name, tool path, affected documentation file(s), severity, and suggested fix action.

Gate: All issues categorized with severity. Proceed only after the gate passes.

Phase 4: REPORT

Goal: Generate human-readable report with actionable fix suggestions. Report facts concisely -- show data, not self-congratulatory descriptions. Target 100% sync score; even one missing entry erodes trust in all documentation.

Step 1: Run the report generator

python3 skills/meta/docs-sync-checker/scripts/generate_report.py --issues /tmp/issues.json --output /tmp/sync-report.md

Step 2: Verify report structure

Report must include these sections:

1. Summary -- Total tools, issue counts by severity, sync score

   sync_score = (total_tools - total_issues) / total_tools * 100
   

2. HIGH Priority: Missing Entries -- For each missing tool, provide the exact markdown row/item to add to the appropriate README file

3. MEDIUM Priority: Stale Entries -- For each stale tool, identify the file and line to remove

4. Files Checked -- List each documentation file with count of tools parsed from it

Step 3: Validate actionability

Every issue in the report must have a concrete suggested fix. No issue should say "review manually" without specifying what to review and where. The fix should enable a single-commit resolution -- tool files and documentation entries should be added/removed together.

Step 4: Report format for missing entries

For each missing skill, generate a suggested table row:

| skill-name | Description from YAML | `skill: skill-name` | - |

For each missing agent, generate a suggested table row:

| agent-name | Description from YAML |

For each missing command, generate a suggested list item:

- `/command-name` - Description from command file

Step 5: Cleanup

Remove any helper scripts and debug outputs created during execution.

Gate: Report generated with actionable suggestions for every issue.

Examples

Example 1: New Skill Missing from README

User created skills/my-new-skill/SKILL.md but forgot to update docs/skills.md. Actions:

1. SCAN discovers my-new-skill in filesystem
2. CROSS-REFERENCE parses docs/skills.md, does not find my-new-skill
3. DETECT flags as HIGH severity missing entry
4. REPORT suggests exact table row to add to docs/skills.md

Example 2: Removed Agent Still Documented

User deleted agents/old-agent.md but agents/README.md still lists it. Actions:

1. SCAN does not find old-agent in filesystem
2. CROSS-REFERENCE finds old-agent in agents/README.md
3. DETECT flags as MEDIUM severity stale entry
4. REPORT suggests removing the row from agents/README.md

Example 3: Batch Changes After Refactor

User created 3 new skills and deleted 2 old ones in a refactoring PR. Actions:

1. SCAN discovers 3 new skills in filesystem, does not find 2 removed skills
2. CROSS-REFERENCE finds 2 stale entries and 3 absent entries in docs/skills.md
3. DETECT flags 3 HIGH (missing) + 2 MEDIUM (stale) issues
4. REPORT provides exact table rows to add and identifies rows to remove

Error Handling

Error: "YAML Parse Error"

Cause: Invalid frontmatter -- missing --- delimiters, tabs instead of spaces, or missing required fields Solution:

1. Check file has opening --- on line 1 and closing --- after YAML block
2. Verify no tab characters in YAML (spaces only)
3. Confirm required fields present: name, description
4. Validate manually: head -20 {file_path} and check syntax

Error: "Documentation File Not Found"

Cause: Expected README file does not exist at expected path Solution:

1. Verify --repo-root path is correct
2. Check that docs/skills.md, agents/README.md, commands/README.md exist
3. If file is legitimately missing, create a placeholder with the expected table/list header
4. Re-run scan after creating placeholder

Error: "No Tools Discovered"

Cause: Wrong --repo-root path, empty directories, or no SKILL.md files Solution:

1. Verify the repo root path points to the correct repository
2. Confirm skills/, agents/, commands/ directories exist and are not empty
3. Check that skill directories contain SKILL.md (not just other files)
4. Run with --debug flag for verbose discovery output

Error: "Markdown Parse Error"

Cause: Table missing separator row, mismatched column counts, or malformed list items Solution:

1. Check table has header row, separator row (|---|---|), and data rows
2. Verify all rows have the same number of pipe-delimited columns
3. For lists, verify consistent format: - /command - Description
4. See references/markdown-formats.md for complete format specifications

---

Deep References

Load when the task requires detailed guidance beyond the phases above.

Signal	Reference
Documentation file matrix, required fields	references/documentation-structure.md
Expected table/list formats per README	references/markdown-formats.md
Sync score formula, severity rules, deprecation	references/sync-rules.md
Before/after examples for doc updates	references/examples.md
CI/CD setup, pre-commit hooks, auto-fix mode	references/integration-guide.md