Skill Improve (orchestrator)
Retrofit the OIAE (Observe/Inspect/Amend/Evaluate) self-improvement loop to existing skills and analyze accumulated feedback to propose evidence-based amendments.
Role definition lives in the subagent
The auditor judgment — OIAE opt-in level assessment, retrofit content
generation, path-discipline grep, feedback pattern detection, amendment
proposal text, and previous-amendment evaluation — lives in the
@agent-skill-improve:skill-improve subagent.
This skill is the orchestrator: it resolves the target skill path,
hands the bundle (SKILL.md, references, feedback log, amendments file) to
the subagent, presents the subagent's proposals to the user for approval,
applies approved amendments, runs the install script, performs the smoke
check, and commits/pushes.
Use this skill when running the full retrofit + amendment cycle. Use
@agent-skill-improve:skill-improve directly when another agent just
needs the auditor judgment without the install/commit flow.
Locating your own reference files
Distinct from "Paths" below (which is about the dev repo root you retrofit
skills inside): this skill's own "$SKILL_HOME"/references/retrofit-checklist.md, and
my-skill-factory's references/skill-improvement-guide.md (a separate
plugin), live at each plugin's install location, not at any relative path
from your working directory. Resolve both in one Bash call before reading
either:
SKILL_HOME=$(ls -d ~/.claude/plugins/cache/hideki-plugins/skill-improve/*/skills/skill-improve 2>/dev/null | sort -V | tail -1)
MSF_HOME=$(ls -d ~/.claude/plugins/cache/hideki-plugins/my-skill-factory/*/skills/my-skill-factory 2>/dev/null | sort -V | tail -1)
echo "$SKILL_HOME"; echo "$MSF_HOME"
Re-derive both in the same Bash call as any later read — shell state doesn't
persist between separate Bash tool calls. If either comes back empty, note
in your output that the corresponding template wasn't reachable rather than
guessing at its content.
Paths
Examples below use <repo-root> for the user's local clone of the skill repository.
Resolve once per session:
- macOS / Linux: typically
~/private/repos/agent-skill-set
- Windows: typically
D:\Shared\agents\my-skills
- Otherwise: ask the user, or run
git -C <any-skill-dir> rev-parse --show-toplevel.
install_skill.py auto-detects the repo via git rev-parse --show-toplevel, so once you
cd <repo-root> you can invoke it with forward-slash relative paths on any platform.
Path discipline (applies to every retrofit and amendment)
When this skill writes new content into an existing skill (Retrospective, Feedback Check,
amendments, file-path references), never introduce hardcoded operator-specific or
OS-specific absolute paths. Forbidden patterns: /Users/<name>/..., /home/<name>/...,
C:\Users\<name>\..., D:\Shared\..., /private/..., or any path that assumes the skill
repo lives at one specific location.
Use <repo-root> (the skill repo root, see "Paths" above), ~ / $HOME, or runtime
resolution (git rev-parse --show-toplevel) instead. Concrete paths are allowed only as
explicitly-framed documentation examples ("on Windows this typically resolves to D:\...").
Pre-install verification. Before running install_skill.py after a retrofit or
amendment, grep the modified skill's source for path leaks:
grep -rn -e '/Users/' -e '/home/' -e '/private/' -e 'C:\\' -e 'D:\\' \
<repo-root>/<skill-name>/ --exclude-dir=feedback
Each remaining hit must be either an example explicitly framed as such, or replaced with
a placeholder / runtime resolution. Apply this check whether the change came from the
retrofit Phase 2 or from an evidence-driven amendment in Phase 3.
Arguments
| Argument | Default | Description |
|---|
--skill | (required) | Skill name or path to the skill directory |
--retrofit-only | false | Only add OIAE components, skip feedback analysis |
--analyze-only | false | Only analyze feedback, skip retrofit |
Workflow
Feedback Check
Before starting Phase 1, look for accumulated feedback on this skill itself:
- If
feedback/log.md exists next to this SKILL.md and has 5 or more entries, read the
last 10.
- If a pattern is apparent (the same issue keyword in 3+ entries, or average rating
below 3), tell the user (in Japanese):
「過去のフィードバックで類似パターンを検出: [簡潔に]。
/skill-improve --skill skill-improve で改善案を分析できます (=自分自身を分析)。」
- Continue with normal execution either way.
If feedback/log.md does not exist, skip silently.
Phase 1: Locate and Read the Skill
-
Resolve skill path:
- If a name is given, check
<repo-root>/<skill-name>/
- If a path is given, use it directly
- Read the skill's SKILL.md
-
Check for existing OIAE components:
version: field in frontmatter### Retrospective section in body### Feedback Check section in bodyfeedback/ directory existence
-
Check for existing feedback:
feedback/log.md — does it exist? How many entries?feedback/amendments.md — any pending amendments?
-
Report current state to user:
"Skill <name>: version [present/missing], Retrospective [present/missing], Feedback Check [present/missing], feedback entries: N"
Phase 2: Retrofit (if components missing)
Skip this phase if --analyze-only is set or all OIAE components are already present.
Read "$SKILL_HOME"/references/retrofit-checklist.md for the step-by-step process and placement rules.
-
Assess opt-in level (None / Observe / Full):
- Full: Skill has a multi-phase workflow, expected >5 uses, complex output
- Observe: Uncertain usage frequency, simpler workflow
- None: One-shot skill, CLI wrapper, deterministic utility
-
Present assessment to user:
"I recommend [level] for this skill because [reason]. Proceed?"
-
Add missing components using templates from "$MSF_HOME"/references/skill-improvement-guide.md:
- Add
version: 1.0.0 to frontmatter if missing
- Add
### Feedback Check section (Full level only)
- Add
### Retrospective section (Observe or Full level)
-
Run the path-discipline grep from "Path discipline" above on the skill's source dir
and replace any non-example hits with placeholders or runtime resolution.
-
Run the install script:
cd <repo-root>
python my-skill-factory/scripts/install_skill.py <skill-name>
-
Smoke check (REQUIRED before reporting completion). Verify the
retrofitted skill actually loads:
claude -p listing confirms <skill-name>:<skill-name> is present in
a fresh session.- Frontmatter
version field reads as the value just written.
- Path-discipline grep over the source dir still returns no non-example
hits.
Code-only review is NOT sufficient — a typo in the new Retrospective
section, a malformed frontmatter, or a missed path leak only surfaces
when the skill is actually loaded. Treat any issue found here as a
smoke-discovered fix to commit before reporting completion.
-
Commit and push:
cd <repo-root>
git add <skill-name>/ my-marketplace/plugins/<skill-name>/ my-marketplace/.claude-plugin/marketplace.json
git commit -m "chore: retrofit OIAE improvement loop to <skill-name> skill"
git push
Phase 3: Analyze Feedback (if log.md exists)
Skip this phase if --retrofit-only is set or no feedback/log.md exists.
Read "$MSF_HOME"/references/skill-improvement-guide.md for pattern detection heuristics and amendment format.
-
Evaluate previous amendments — If feedback/amendments.md exists with entries in applied — monitoring status:
- Read log entries dated after the amendment
- Check if the specific issue pattern recurred
- Check if ratings improved
- Update status to
effective, ineffective, or insufficient data
- For ineffective amendments, suggest
git revert <commit>
-
Read all feedback — Read feedback/log.md in full
-
Identify patterns using these heuristics:
- Same issue keyword in Corrections/Issues across 3+ entries → recurring problem
- Average rating below 3.0 over last 10 entries → general underperformance
- Declining ratings over time → skill degrading
- Outcome distribution >30% partial-success or failure → structural issues
- Issues clustering after a version bump → recent amendment may have caused problems
-
Read current skill — SKILL.md + relevant references to understand current instructions
-
Propose amendments — For each identified pattern:
- What to change (file path, section)
- Why (cite specific feedback entries by date as evidence)
- The proposed text change
- Suggested version bump (patch or minor)
-
Present to user — Show all proposed amendments with supporting evidence for approval
-
Apply approved changes:
- Edit the skill files
- Bump version in frontmatter
- Run the path-discipline grep from "Path discipline" above on the modified skill;
replace any non-example hits with placeholders or runtime resolution.
-
Record amendment — Append to feedback/amendments.md using format from "$MSF_HOME"/references/skill-improvement-guide.md
-
Install:
cd <repo-root>
python my-skill-factory/scripts/install_skill.py <skill-name>
-
Smoke check (REQUIRED before commit/push). Verify the amended skill
actually loads:
claude -p listing confirms <skill-name>:<skill-name> is present
in a fresh session.- Frontmatter
version field reads as the bumped value.
- Path-discipline grep over the source dir returns no non-example hits.
- For amended skills that have scripts/lib (e.g., deno-backed wrapper
skills), the script files at least syntactically parse / type-check
via a dry
--check or equivalent (where the runtime supports it).
If smoke fails, fix the underlying issue and re-install before
committing. Never ship an amendment that fails its own smoke check —
that's exactly the regression class AMD-003 is preventing.
-
Commit and push:
cd <repo-root>
git add <skill-name>/ my-marketplace/plugins/<skill-name>/ my-marketplace/.claude-plugin/marketplace.json
git commit -m "fix: improve <skill-name> skill based on feedback (AMD-NNN)"
git push
Phase 3b: Ecosystem Drift Check
Skip only if --retrofit-only is set. Runs even when the skill has no feedback log:
a skill nobody has complained about can still be written against an outdated
ecosystem (a retired model ID, a superseded frontmatter rule, a hand-rolled check
that a CLI now does natively).
- Read
"$MSF_HOME"/references/ecosystem-practices.md, the factory's knowledge
base of current Claude Code / Codex / model / authoring practices. If it is
missing or empty, say so in one line and skip this phase.
- Compare the target skill against each item that applies to its type. Flag only
concrete contradictions or missed practices, never general style.
- Derive proposals only from items in that file, never from your own recollection
of the ecosystem. Every item there was sourced and user-approved; recollection was
not. Items with a
verified date older than 90 days are reported as stale, not
proposed.
- Present, apply, record, install, smoke, and commit exactly as in Phase 3
steps 6–11. Use
Source: ecosystem-practices in the amendment, with the item and
its source URL as Evidence, and a commit message of the form
fix: align <skill-name> with current ecosystem practices (AMD-NNN).
When Phase 3 also produced amendments, present both sets together once.
Phase 4: Report
Summarize what was done:
- OIAE components retrofitted (if any)
- Previous amendments evaluated (status updates)
- Patterns found in feedback (if any)
- Ecosystem drift found (if any)
- Amendments applied (if any)
- Current skill version
Retrospective
After Phase 4 (Report) completes, reflect on this run of skill-improve itself:
-
Consider: were there mid-session corrections (rejected amendments, scope changes,
wrong target skill, install/commit failures, missed path leaks)?
-
Ask the user (in Japanese): 「今回の改善作業のフィードバック (1-5の評価、気になった点、または何もなければEnter)」
If the user provides a rating < 5, ALWAYS follow up with:
「なぜその評価ですか？ (改善のために具体的に教えてください)」
Record the response verbatim as Rating reason. A rating without the
"why" loses the strongest improvement signal — never skip this follow-up
for ratings 1-4, even in auto-mode.
-
If the user provides feedback OR if corrections/issues actually occurred:
a. Create feedback/ next to this SKILL.md if it does not exist (resolve the
directory via git rev-parse --show-toplevel from this skill's source dir,
then append /skill-improve/feedback/).
b. Read feedback/log.md (create with # Feedback Log header followed by a
blank line and the comment
<!-- Append new entries at the top. Do not edit previous entries. -->
if it does not exist).
c. Prepend a new entry directly after the header, using the format from
"$MSF_HOME"/references/skill-improvement-guide.md:
## <ISO-8601 timestamp>
- **Skill Version**: <version from this file's frontmatter>
- **Task**: <which target skill, retrofit / analyze / both>
- **Outcome**: success | partial-success | failure | error
- **Rating**: <N>/5 (or "—" if not provided)
- **Rating reason**: <user's verbatim response to the WHY follow-up, or "—" if rating was 5 or not provided>
- **Corrections**: <mid-session corrections, or "none">
- **Issues**: <specific problems, or "none">
- **User Note**: <user's verbatim feedback, or "—">
---
d. Confirm in one short Japanese sentence.
-
If the user skips AND no corrections or issues occurred, end without recording.
Behavior Scenarios
BDD spec lives in "$SKILL_HOME"/references/scenarios.feature (resolve
$SKILL_HOME per "Locating your own reference files" above). Read only when
auditing or amending this skill (e.g., via /skill-improve --skill skill-improve); not needed for normal execution.
References
Read these on-demand only — they are not auto-loaded. Resolve $SKILL_HOME
/ $MSF_HOME per "Locating your own reference files" above first.
"$SKILL_HOME"/references/scenarios.feature — Gherkin BDD spec for this
skill. WHEN TO READ: only when auditing or amending the skill itself
(e.g., via /skill-improve --skill skill-improve). Never during normal
execution."$SKILL_HOME"/references/retrofit-checklist.md — Step-by-step checklist for adding OIAE
components with placement rules. WHEN TO READ: only in Phase 2
(retrofit) when OIAE components are missing from the target skill."$MSF_HOME"/references/skill-improvement-guide.md — OIAE protocol,
log format, amendment format, templates, pattern detection heuristics.
WHEN TO READ: in Phase 2 when looking up template wording, and in
Phase 3 when applying pattern-detection heuristics or formatting an
amendments.md entry."$MSF_HOME"/references/ecosystem-practices.md — the factory's sourced,
user-approved knowledge base of current Claude Code / Codex / model / authoring
practices. WHEN TO READ: in Phase 3b only.