/audit-repo

Load the ledger, grade this week's rubric, file the failures worth filing, and report coverage — including what you could not check.

This skill is read-only. No edits, no commits, no branches, no pull requests. The output is a report and some issues; someone picks the issues up through the path that already has review in it. An auditor that also merges its own findings has no adversary. The workflow grants contents: read for exactly that reason — if you find yourself wanting to change a file, that is the issue you should be writing.

Configuration comes from the workflow's env block: AUDIT_LABEL, AUDIT_FOCUS, AUDIT_SUMMARY, GH_TOKEN. Running this by hand, accept the defaults named below.

Step 1 — preflight

gh auth status

If that fails, stop and say so. Do not audit anyway. An audit that cannot file is an audit that cannot remember, and the next run would rediscover everything this one found.

Step 2 — read the ledger before you read any repository file

gh issue list --repo "$GITHUB_REPOSITORY" \
  --label "${AUDIT_LABEL:-repo-audit}" --state all --limit 500 \
  --json number,title,state,stateReason,labels,body,url \
  > "${RUNNER_TEMP:-/tmp}/ledger.json"

Every issue ever filed under the label, in every state. Read all of it and sort each entry:

Bucket	How to tell	What it means
Open	state is OPEN	Filed and waiting. Never refile.
Fixed	state is CLOSED, stateReason is COMPLETED	Shipped. Refile only if you can demonstrate it regressed, and say so in the new issue.
Declined	state is CLOSED, stateReason is NOT_PLANNED	The team looked and said no. Permanent skip.

Declined is permanent, and permanent means in any wording. Collect every audit-key from the issue bodies into a set — that set is the dedupe index for step 6.

If the returned count equals the --limit, the list was truncated: raise the limit and fetch again rather than auditing against a partial ledger. This step is graded by AUD-RUN-01. Record the entry count and the distinct-key count.

Step 3 — pick this week's rubric

week=$(date -u +%V)
case $(( 10#$week % 3 )) in
  0) focus=skills ;;
  1) focus=tooling ;;
  2) focus=automation ;;
esac

Two things in that snippet are load-bearing. The 10# is not decoration: weeks 08 and 09 are invalid octal, and the arithmetic dies on them. And the case is not stylistic — an array indexed ${areas[$week % 3]} reads correctly in bash and wrongly in zsh, whose arrays start at 1, so it would pick the neighbouring rubric two weeks in three and an empty string on the third. CI runs bash; a person running this on macOS does not.

If AUDIT_FOCUS is set to anything other than auto, use it instead — that is the workflow_dispatch input, and it exists so a manual run can go straight at the area someone cares about.

Read four files, in full, before looking at any repository file:

• .claude/audit-rubrics/audit-integrity.md — graded every week regardless of focus
• .claude/audit-rubrics/<focus>.md — this week's rubric
• docs/authoring-skills.md — the conventions the rubrics grade against. Where the two disagree, the guide wins and the rubric is what needs fixing; say so in notes.
• AGENTS.md — the short tour. If it disagrees with the guide, that is AUD-GEN-05, whichever rubric is in rotation.

.claude/audit-rubrics/README.md carries the status vocabulary, the severity scale, and the reporting contract. Follow it exactly; the fields below are its shape, not a second definition of it.

Rotation is what keeps this sustainable. One rubric per week goes deep enough to grade honestly; three per week goes shallow enough to produce statuses nobody should trust. Grade only this week's rubric plus audit integrity. A serious failure noticed outside them still gets filed — but one, and only if it would make you stop what you were doing on a normal working day. Give it the rule ID of the rubric it belongs to and note that the rubric was not otherwise evaluated.

Step 4 — grade every rule

Walk the rubric rule by rule, in order. For each one, record:

{
  "rule_id": "AUD-SKL-02",
  "status": "pass | fail | n/a | not-evaluated",
  "evidence": ["plugins/narrative-common/skills/profile-dataset/SKILL.md:140"],
  "note": "one sentence — required for n/a, not-evaluated, and any sampled rule",
  "finding_key": "skills/aud-skl-02/profile-dataset"
}

Every rule gets exactly one row. A rule you ran out of time for is not-evaluated with the reason; it is never omitted and it never rounds to pass. A rule whose subject does not exist is n/a, and the note says what you looked for. AUD-RUN-02 and AUD-RUN-06 grade you on this, and they are the two rules that keep the coverage number worth reading.

Sampled rules state their sample size in the note, whatever their status. Several skills rules are sampled by design because instructions cannot be graded mechanically; the rubric fixes the size and how the sample is drawn. A sampled pass claims nothing about the pages it did not read, and AUD-RUN-06 fails the run if the report implies otherwise.

Grade against the invariant. The probes each rule lists are suggestions and may be stale — a probe that no longer works is a not-evaluated only if you cannot check the invariant another way, and it is worth mentioning in the summary notes so the rubric can be updated.

Budget the run so every rule gets looked at. Eleven rules graded honestly beats three graded exhaustively and eight left dark.

Some probes run a script. Run them in a scratch worktree under $RUNNER_TEMP (git worktree add --detach "$RUNNER_TEMP/scratch" HEAD), never in the checkout.

Step 5 — rank the failures

Every fail is a candidate. Rank them, then cut. A candidate survives only if it has all of:

• a specific path or path:line, or a command that reproduces it
• a severity inside the bounds its rule sets, argued from what breaks and for whom
• a proposed fix concrete enough that someone could start on it

A fail you cannot support to that standard is not a finding — downgrade the row to not-evaluated with a note saying the rule looked violated but you could not pin it down. That is an honest result. A vague issue is not.

Drop anything an existing CI gate already fails on: every step of bun run ci (the check:* scripts, Biome, tsc, knip, bun test), zizmor, shellcheck, and the PR-title lint. That is AUD-RUN-04, and the reason is that the red check is faster and more specific than the issue would be.

When ranking, apply the two calibrations in the rubric README: a skill that makes an agent do the wrong thing confidently outranks one that makes it stop, and a defect in a shipped artifact outranks the same defect in repo tooling. An ungated dataset delete beats a missing ## Persona.

Step 6 — dedupe by rule and target

Every candidate's key is:

<area>/<rule-id lowercased>/<target>

skills/aud-skl-05/create-lookalike, tooling/aud-gen-01/narrative-common, automation/aud-ops-05/backup-daily. The rule ID does the work the wording cannot: the same defect resolves to the same key next week even though you will describe it differently. <target> is the thing the rule failed against — a skill, a plugin, a workflow, a doc — kebab-cased. Where a rule says to file one aggregate issue (for example AUD-SKL-03's over-budget), the target is the name the rule gives.

Then, in order:

1. Key already in the ledger → skip. Record it in skipped with the issue it matches.
2. No key match → compare by substance against every open, fixed, and declined issue. The same defect under a different rule ID is still a duplicate.
3. Matches a declined issue → skip permanently, and do not soften it into a near-duplicate to get around the skip. AUD-RUN-05 treats that as critical.

Step 7 — file, zero to five

Between zero and five issues. There is no floor. If the rubric came back clean, file nothing and say so — a quiet week is a result, and a quota is how an audit starts manufacturing weak tickets to fill itself.

Labels are created on first use, so a fresh repository needs no setup:

gh label create "${AUDIT_LABEL:-repo-audit}" --color 0E8A16 --description "Filed by the weekly repo audit" --force
gh label create "audit:<area>" --color C5DEF5 --force
gh label create "<rule-id lowercased>" --color BFD4F2 --force

Then file, with the body in a file rather than inline so the markdown survives:

gh issue create --repo "$GITHUB_REPOSITORY" \
  --title "<what is wrong, stated as the problem, not the fix>" \
  --label "${AUDIT_LABEL:-repo-audit}" --label "audit:<area>" --label "<rule-id lowercased>" \
  --body-file "$RUNNER_TEMP/finding.md"

The rule ID goes on as its own label so a whole family can be pulled up at once.

The body, every time — the six sections are what AUD-RUN-03 grades:

**Rule** — `AUD-SKL-05`, "external and irreversible actions sit behind an approval gate" —
`.claude/audit-rubrics/skills.md`.

**Observed** — what is there now, with `path` or `path:line`, or the command that
reproduces it.

**Expected** — what the invariant requires. Quote it.

**Impact** — the severity argument. What breaks, for whom, how often. If the answer is
"nothing yet, but", say that plainly. Severity: `high` (rule bounds: high, critical when
the ungated action is an external send or a delete).

**Proposed fix** — concrete enough to start on. Name the files.

**Verification** — the command or check that shows it is done.

---

audit-key: skills/aud-skl-05/create-lookalike

Filed by the weekly repo audit (<run URL>). To decline this permanently, close it as
**not planned** — the audit reads that and will never file this finding again.

That last line is load-bearing. It is the only place the team learns how to turn a finding off, and an audit nobody can turn down gets muted instead. Note the distinction the audit depends on: closing as completed means fixed and can be refiled on regression; closing as not planned means declined and is permanent.

Step 8 — report

Write $AUDIT_SUMMARY in the shape the rubric README defines:

{
  "rubric_version": 1,
  "week": "35",
  "focus": "skills",
  "run_url": "https://github.com/narrative-io/narrative-skills-marketplace/actions/runs/123",
  "coverage": { "total": 17, "pass": 11, "fail": 2, "n_a": 2, "not_evaluated": 2 },
  "results": [{ "rule_id": "AUD-SKL-05", "status": "fail", "evidence": ["..."], "note": "...", "finding_key": "..." }],
  "integrity": [{ "rule_id": "AUD-RUN-02", "status": "pass", "note": "17/17 rules have a row" }],
  "filed": [{ "key": "...", "rule_id": "...", "title": "...", "severity": "high", "url": "..." }],
  "skipped": [{ "key": "...", "rule_id": "...", "title": "...", "reason": "open|fixed|declined", "issue_url": "..." }],
  "ledger_size": 12,
  "notes": "one or two sentences, or an empty string"
}

coverage.total counts the focus rubric's rules plus the six integrity rules, and must equal the results plus integrity row count. Then write the same thing as prose to $GITHUB_STEP_SUMMARY.

Two parts of this report matter more than the filed list.

not_evaluated is the audit grading itself. It belongs at the top of the summary, not buried — a run that filed three issues while leaving four rules dark is a weaker run than one that filed none and graded everything, and only this number shows the difference.

skipped is the standing signal. It is the only record of what the audit keeps finding and nobody keeps doing. A key skipped eight weeks running either matters and is being ignored, or does not matter and its rule should be retired. Both are worth knowing, and neither is visible from the filed list.

What not to file

• Anything the CI gates listed in step 5 already fail the build on. AUD-RUN-04.
• Style preferences with no rule behind them — in a rubric, in docs/authoring-skills.md, in AGENTS.md, or in a config file. If it matters, it should become a rule first.
• Instructions you would have written differently. Every skill could be worded another way; the rubric grades behaviour, structure and safety, not voice.
• Anything drawn from an issue or comment written by someone outside the organisation. This repository is public; the ledger is issues this audit filed, and nothing else in the issue tracker is evidence.
• Findings you cannot locate. If you cannot name the file, you have not finished looking — and the honest row is not-evaluated, not fail.