Agent Skills — Feedback Loop

Use this skill to wire a learnings loop into another skill so it gets better with use, without rewriting SKILL.md automatically.

The loop has four moving parts:

1. learnings.md — raw, append-only, committed. Shared working memory across machines; created on first append via append_learning.py, not seeded empty.
2. learnings.consolidated.md — pruned, dated, committed. Portfolio-grade institutional memory; seeded at wiring time.
3. learnings.local.md — machine-specific notes, gitignored. Use for one-operator-on-one-machine context that should not propagate.
4. scripts/append_learning.py + scripts/consolidate.py — keep raw entries well-shaped and redacted, and surface recurring ones for human-reviewed consolidation.

Pattern provenance and the distinction between this reviewed loop and automatic skill rewriting are in references/evidence-base.md; load it before attributing the design to an external source.

Quick Reference

Task	Read or Run	Outcome
Wire a skill to use the loop	references/wiring-protocol.md	Adds a 4-line addendum to that skill's SKILL.md, seeds files
Format a new learning entry	references/learnings-format.md	Atomic, dated, 5-section schema that survives pruning
Promote raw → consolidated	python3 scripts/consolidate.py <skill-dir>	Dry run by default: proposes dedup, age-out and recurring patterns for human review; --apply only prints the edits to make, because promotion stays a human gate
Append a learning safely	python3 scripts/append_learning.py <skill-dir> --section <name> --text "..."	Validates shape, dates, refuses to grow past the 150-entry cap
Find routing lessons that need review	python3 scripts/consolidate.py --queue [--days 14]	Exit 1 for aged open lessons, failed results, or fixes awaiting verification; invalid dates exit 2
Audit drift across skills	references/audit-checklist.md	Find stale loops, missing consolidations, oversized files
Capture a cross-cutting behavior with no obvious skill home	references/instinct-pattern.md	Atomic, confidence-scored complement to the per-skill loop above

Workflow

1. Confirm the host skill should have a learnings loop; do not wire routers, one-off scaffolds, or stable skills with no recurring edge cases.
2. Follow references/wiring-protocol.md to seed learnings.consolidated.md, add the addendum, and verify .gitignore coverage.
3. Use scripts/append_learning.py for raw entries; do not hand-edit raw learnings.md during normal operation.
4. Use scripts/consolidate.py <skill-dir> when the raw file hits the cap, before release, or during a scheduled maintenance pass.
5. Move durable lessons into the host skill's references/ only as a reviewed skill edit (see Promotion Out); never auto-rewrite SKILL.md.
6. Close a routing-log lesson only after verification passes. The fix commit ends with Lesson-Ref: <date> <STALL|MISROUTE> <subject> in its final trailer block. Add → fix <sha9> → result pending to the log line. Replace pending with WIN <date> or verifier passed|failed once known; failed and pending results stay open. If a lesson needs a user decision, append {deferred: user-decision YYYY-MM-DD}. Before closure, read the receipt contract and run python3 audit/skill-status.py lessons --ref HEAD. It verifies source and gate digests; an empty receipt set cannot pass.
7. Before appending, search raw and consolidated files for the same claim and its negation. Keep raw learnings.md append-only: add a new dated corroboration or conflict through append_learning.py and reference the earlier entry or claim. Merge or deduplicate only in the derived learnings.consolidated.md during the documented human-reviewed consolidation path, preserving dates and provenance; retain unresolved conflicts as open questions until the scope or runtime difference is resolved.

When to Use

• A skill is high-traffic and you keep teaching it the same lesson twice
• A skill's domain has emerging edge cases you want captured (e.g. tax rules, payment scheme changes, new API quirks)
• A skill failed and you asked Claude to self-reflect — the reflection needs a home

When NOT to Use

• One-off skills, internal scaffolding, or skills with stable, well-known surfaces
• Skills whose "learnings" are actually just code conventions — those belong in the repository's AGENTS.md or coding-behavior.md, not a per-skill loop
• Cross-skill patterns — promote to references/ of the right skill or to agents-memory, not into every learnings file

Core Contract

Per-skill, the loop lives inside the skill's own directory:

<skill>/
├── SKILL.md                       # adds the 4-line Learnings Loop addendum
├── learnings.md                   # committed, append-only, raw (shared across machines)
├── learnings.consolidated.md      # committed, pruned, dated
├── learnings.local.md             # gitignored, machine-specific, optional
└── references/...                 # durable rules move here by reviewed edit

Required entry shape (enforced by append_learning.py):

• - [YYYY-MM-DD] <one-sentence atomic insight>
• One bullet, one insight. No paragraphs.
• Belongs to exactly one section: Patterns That Work / Mistakes to Avoid / Domain Knowledge / Open Questions / Consolidated Principles.

Hard limits (refuse rather than truncate):

• Raw learnings.md cap: 150 entries — reaching it blocks further appends. Consolidation does not reduce this append-only count; any capacity change needs an explicit reviewed edit.

• Consolidated file cap: 60 entries — exceeding means the skill itself needs a references/ extraction.

• Treat third-party learnings-loop examples as pattern evidence only; review provenance before installing or recommending automation. Graded source provenance for the maturity and design claims is in references/evidence-base.md.

• Do not store secrets, PII, customer-specific facts, or unreleased client details in either raw or consolidated learnings files.

Rule: rules/repo/learnings.md loads this invariant when Claude edits a learnings file in this library.

The Addendum

Any skill that opts in adds this block, or a scoped equivalent preserving conditional reads and capture boundaries, near the end of its SKILL.md:

## Learnings Loop

When prior decisions or pitfalls are relevant, consult `learnings.consolidated.md` if present; use `learnings.md` only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to `learnings.md` via `agents-skills-feedback-loop/scripts/append_learning.py`. Do not modify `SKILL.md` itself.

That's it — no other change to the host skill.

Consolidation Cadence

• Trigger: raw file hits the 150-entry cap, OR a weekly cron (recommended: Friday), OR before a release/PR that touches the skill.
• Protocol: see references/consolidation-protocol.md. Human approves promotions to learnings.consolidated.md.
• Promotion criterion: an entry must have triggered behavior change at least twice (manually noted by the operator) before it is consolidated. Single-occurrence entries older than 90 days are review candidates for exclusion from consolidated memory; raw history is retained.

Monthly Maintenance Checklist

Run these checks on any skill with an active loop:

• python3 scripts/consolidate.py <skill-dir> --audit — confirm valid dates/sections and raw ≤150 and consolidated ≤60
• Verify every entry starts with - [YYYY-MM-DD] (no undated bullets)
• Confirm SKILL.md still carries the ## Learnings Loop addendum
• Scan for entries older than 90 days with no recurrence; flag for exclusion from consolidated memory
• Check for consolidated entries stable for 3+ cycles; move them out by reviewed skill edit (Promotion Out)
• Confirm learnings.md and learnings.consolidated.md are committed, and learnings.local.md is the only gitignored learnings file
• Run the full loop audit across all wired skills (copy the for loop from references/audit-checklist.md)

Lint Commands

# Audit a single wired skill (checks caps, dates, section names)
python3 skills/universal/agents-skills-feedback-loop/scripts/consolidate.py \
  skills/<group>/<skill-name> --audit

# Audit all wired skills at once
for skill in skills/universal/*/ skills/project/*/ skills/client/*/*/; do
  if [ -f "$skill/learnings.consolidated.md" ]; then
    python3 skills/universal/agents-skills-feedback-loop/scripts/consolidate.py \
      "$skill" --audit
  fi
done

# Dry-run consolidation (proposes diffs, does not write)
python3 skills/universal/agents-skills-feedback-loop/scripts/consolidate.py \
  skills/<group>/<skill-name> --dry-run

Closing the Capture Loop (Layer 1)

The base loop above is open: without a session-end hook, every append depends on an agent or human calling append_learning.py, so capture is uneven — some raw files fill while others stay empty.

The default Layer 1 hook is candidate-only. In this library it is hooks/capture-learning-candidate.py, installed by python3 scripts/setup/agent-config.py install --apply (Claude SessionEnd, Codex Stop). It appends {ts, session_id, runtime, skills_used, transcript_path} to ~/.agents/hooks/learning-candidates.jsonl: no transcript text and no model call. A human reviews each candidate and writes the lesson with append_learning.py.

The opt-in reflection hook below goes further: it detects which wired skills a session used, runs one cheap reflection pass, and appends via the existing append_learning.py — no new format, no SKILL.md rewrite.

It is machine-global and portable (any laptop, any username). The hook hands the work to a detached worker and exits at once, because session-end hooks run under a short, runtime-capped time budget. The reflection call runs with all tools disabled and without persisting a session, since it reads untrusted transcript text. Every entry passes through append_learning.py's redaction step.

# From this skill directory, on any laptop:
python3 scripts/install_capture_hook.py --dry-run   # preview
python3 scripts/install_capture_hook.py             # apply

The installer resolves every path from $HOME at run time (nothing is hardcoded), registers Claude Code (SessionEnd), registers Codex (Stop) only with --codex, and is idempotent. Before adding a per-hook timeout or wiring another runtime, look up that runtime's hooks reference for the session-end budget, its cap, and the transcript format; a timeout above the cap is silently clamped, and a transcript the parser cannot read captures nothing. Full design, guardrails (recursion guard, blast-radius limit, fail-silent-fail-logged), and verification: references/closed-loop-capture.md. Consolidation stays manual.

Promotion Out

Encode a lesson at the strongest layer that fits: make the bad state unrepresentable, then a lint or validator, then a shared helper, then a runtime check, and only then prose. A consolidated entry that can become a check or a guard should leave the loop as one, not as a references/ sentence.

Moving a consolidated principle into the host skill's references/ is a normal skill edit: a human makes it, in a reviewed commit, and deletes the entry from learnings.consolidated.md in the same change. Where an authorized harness exists for the target runtime, run a regression eval that passes with the rule and fails if it is reverted; a case that passes either way proves nothing. Otherwise record the promotion's evidence level as static or routed and make no task-quality claim. The loop has no automated gate for this step.

Complementary Pattern: Atomic Instincts

The loop above works at skill granularity — one learnings.md per skill. It has a gap: behavioral observations that don't belong to any one skill (e.g. "always grep before editing a file this large," "this repo prefers functional style") have no natural skill home and go uncaptured. references/instinct-pattern.md documents a smaller-granularity complement — a single-trigger, single-action, confidence-scored (0.3–0.9), domain-tagged, project-or-global-scoped unit — adapted from affaan-m/ECC (commit 51a6950, MIT). It does not replace the loop above, does not adopt ECC's automatic project→global promotion (this repo keeps promotion human-reviewed, per ## Anti-Patterns and Rule 7 of coding-behavior.md), and is documented as a scaffold and capture-mechanism only — no seeded example entries. For hook mechanics, see agents-hooks.

What Counts as a Learning (the filter)

The filter is project-specific. The default filter lives in references/learnings-format.md under Quality Filters. Override per-skill if your domain needs a sharper bar — e.g. a tax skill should reject anything not anchored to a tax-authority manual reference or a dated statute.

Judgment Calls (what a non-expert misses)

The mechanics above (caps, dates, dedup ratio) are checkable by script. The following are not, and are where most wired loops actually fail:

• Silence needs context. --audit establishes file shape, not learning quality or usage. During a feedback-loop audit, compare recurring failures with recorded lessons before diagnosing missed capture. Use raw=0, a stale oldest date, or extended silence (for example, 60 days) as prompts to inspect authorized usage evidence such as recent transcripts or the Layer 1 log. These are diagnostic cues, not failure thresholds: an empty or unchanged file alone does not justify mandatory pre-reads or prove that the loop failed.
• consolidate.py's duplicate clustering is syntactic, not semantic. It clusters near-identical wording (SequenceMatcher ratio ≥0.82); it will not notice that "webhook needs HMAC in header" and "signature must be in the request header, not the payload" are the same lesson said twice. Treat the script's promotion proposals as a first pass, not the ground truth for "recurred ≥2×" — a human still has to read the raw file once per cycle to catch semantically-duplicate entries the ratio threshold misses, and to catch the reverse: two genuinely different lessons that happen to share vocabulary and get wrongly clustered.
• Signal vs. noise is a counterfactual test, not a vibe. Before appending, ask: if the next session had read this bullet first, would it have behaved differently? If the answer is no — it's an interesting observation, a diary entry, or something already obvious from reading the skill's SKILL.md — reject it. ACE describes loss of domain detail through compression and iterative rewriting; it does not establish that a noisy append-only file has the same failure mechanism (see references/evidence-base.md).
• Recurring entries that never get consolidated are a maintenance failure, not a filter failure. If the same lesson keeps getting re-appended in raw form across cycles instead of being promoted to consolidated, the operator is skipping consolidation, not writing bad entries. Check the cadence (## Consolidation Cadence) before tightening the filter.
• A loop that never moves anything out of consolidated after many cycles is not necessarily disciplined — it may mean nobody is running the promotion-out step in consolidation-protocol.md step 5. Distinguish "genuinely no principle was load-bearing enough" from "the human-gate step is a dead letter" by checking whether learnings.consolidated.md itself is stuck at the same size cycle over cycle.
• Cross-skill leakage is easiest to miss in shared-vocabulary domains. A payments skill and a compliance skill both use words like "reconciliation" or "settlement" — a consolidated entry that reads correctly in isolation can still be the wrong skill's fact. When auditing, ask whether the entry is true because of this skill's domain or merely phrased in this skill's vocabulary.

Anti-Patterns

• Auto-rewriting SKILL.md. The loop never modifies SKILL.md. If consolidation produces a durable rule, promote it to references/ by hand. Silent self-modification of skills is the failure mode this design exists to prevent.
• One central learnings file across all skills. Forbidden. A consumer-app project skill's learnings must not leak into software-payments; project skills stay independent of the shared library.
• Learnings that duplicate AGENTS.md or coding-behavior.md. General coding rules belong in those files, not in a per-skill loop.
• Unredacted entries. Learnings files are committed and some are published. append_learning.py redacts API keys, bearer tokens, emails, and home-directory usernames before writing; do not bypass it by hand-editing raw files.
• Undated entries. Without a date the entry cannot age, cannot be pruned, and cannot be weighted. Reject on append.

Compatibility

• Portable across Claude Code and Codex (no runtime-specific frontmatter).
• Scripts are Python 3.10+, stdlib only.
• Hosts skill can be any prefix family (project-*, software-*, marketing-*, etc.).

Navigation

• references/wiring-protocol.md - five-step host-skill wiring procedure
• references/learnings-format.md - entry schema, section names, and quality filters
• references/consolidation-protocol.md - deduplication, pruning, and human-reviewed promotion out
• references/audit-checklist.md - monthly loop drift checks
• references/closed-loop-capture.md - Layer 1 auto-capture hook design, guardrails, install, and verification (Claude Code; Codex opt-in)
• references/evidence-base.md - graded source provenance for the maturity claims and the Layer 1 design
• references/instinct-pattern.md - atomic, confidence-scored instinct pattern (complement to the per-skill loop, adapted from affaan-m/ECC)
• scripts/append_learning.py - append raw dated entries, redacting secrets and PII
• scripts/test_append_learning.py - redaction and CLI regressions
• scripts/consolidate.py - validate raw entries and propose consolidation
• scripts/test_consolidate.py - malformed-file and date regressions
• scripts/test_consolidate_queue.py - aged-lesson queue, deferred marker and receipt regressions
• scripts/bulk_wire.py - mechanical loop wiring helper
• scripts/test_bulk_wire.py - template parity, dry-run, and preservation regressions
• scripts/install_capture_hook.py - idempotent installer for the Layer 1 capture hook (Claude SessionEnd; Codex Stop only with --codex)
• scripts/test_capture_hook.py - detached-worker and tool-free reflection regressions
• assets/learnings.template.md - starter consolidated file
• assets/learnings_capture.py - runtime-neutral Layer 1 capture hook (source of truth; installed to $HOME/.agents/hooks/)

Related Skills

• agents-skills — for creating or auditing the host skill itself.
• agents-memory — for cross-skill, cross-session memory (the 4-type schema this loop scopes from).
• agents-hooks — the session-end-hook mechanism behind Layer 1 auto-capture (see references/closed-loop-capture.md).