Skill Evolver
Analyze skill execution traces to discover issues, identify improvement opportunities, and apply fixes to skill files.
Trace Format
Traces are JSON with this structure:
{
"id": "uuid",
"request": "user's original request",
"skills_used": ["skill-name"],
"success": true/false,
"total_turns": 2,
"total_input_tokens": 5000,
"total_output_tokens": 200,
"duration_ms": 7000,
"steps": [
{"role": "assistant", "content": "...", "tool_name": null},
{"role": "tool", "tool_name": "...", "tool_input": {}, "tool_result": "..."}
],
"llm_calls": [
{"turn": 1, "stop_reason": "tool_use", "input_tokens": 2500, "output_tokens": 50}
]
}
Workflow
This skill can receive two types of input (at least one required):
- Traces: Execution trace data from real skill runs — provides data-driven problem discovery
- Feedback: User-written improvement suggestions — provides directed guidance for changes
When both are provided, combine insights: use traces to validate/discover issues and feedback to prioritize and guide fixes.
Step 1: Analyze Inputs
If traces are provided, run the analysis script:
scripts/analyze_traces.py <traces.json> [--skill <name>] [--format json|text]
Output includes:
- Success rate
- Average turns, duration, tokens
- Common issues and warnings
- Recommendations
If feedback is provided, identify the user's improvement goals and map them to actionable changes.
If both are provided, cross-reference: does the feedback align with trace-discovered issues? Use feedback to prioritize which trace-identified problems to fix first.
Step 2: Extract Issue Details
For failed or problematic traces, extract full context:
scripts/extract_issue_context.py <traces.json> --failed
scripts/extract_issue_context.py <traces.json> --trace-id <id> --show-llm
scripts/extract_issue_context.py <traces.json> --high-turns
Skip this step if only feedback was provided (no traces).
Step 3: Identify Root Causes
Map issues to skill components using references/issue-patterns.md:
| Issue Type | Likely Fix Location |
|---|
| execution_failure | scripts/, error handling |
| high_turn_count | SKILL.md clarity, add examples |
| tool_errors | scripts/, input validation |
| high_token_usage | SKILL.md verbosity, progressive disclosure |
| repeated_tool_calls | SKILL.md decision trees |
For feedback-only input, map the user's suggestions directly to the appropriate skill components.
Step 4: Apply Fixes
Read the target skill and apply changes based on analysis:
- For script errors: Fix scripts, add validation, improve error messages
- For efficiency issues: Add examples, decision trees, clearer instructions
- For token issues: Reduce SKILL.md, move content to references/
- For trigger issues: Update frontmatter description
- For feedback-guided changes: Apply the user's specific suggestions
Scope constraints — strictly follow:
- Only modify the target skill's existing files (SKILL.md, scripts/, references/)
- Do NOT create new reference files, templates, or guides
- Do NOT search the web for domain-specific content
- Do NOT generate CHANGELOG, improvement reports, or other extra deliverables
- The evolved skill files themselves are the sole deliverable
Quick Reference
Issue Severity Levels
- high: Failures, max_tokens, tool errors → Fix immediately
- medium: High turns, high tokens, retries → Optimize
- low: Long duration → Consider optimization
Key Metrics Thresholds
| Metric | Warning | Action |
|---|
| success_rate | <90% | Review failures |
| avg_turns | >4 | Simplify workflow |
| avg_tokens | >30000 | Reduce context |
| duration_ms | >60000 | Optimize scripts |
Common Fixes
Low success rate:
- Add error handling in scripts
- Add input validation
- Clarify ambiguous instructions
High turn count:
- Add decision tree
- Provide more examples
- Use scripts for multi-step operations
High token usage:
- Reduce SKILL.md lines (<500)
- Move details to references/
- Remove redundant examples