Check Agent Skills for documentation structure and Python script behavior, then score quality and security posture. Get grades and improvement priorities, and use pass criteria in pre-commit checks or continuous integration, which runs checks automatically.
Validate, test, and score the quality of skills within the claude-skills ecosystem. Comprehensive meta-skill: structure validation, Python script testing (syntax + imports + runtime + output format), multi-dimensional quality scoring with letter grades and tier classification (BASIC/STANDARD/POWERFUL). Use when authoring a new skill, auditing existing skills for tier promotion, setting up pre-commit hooks for skill quality, or integrating skill QA into CI.
Review the instructions the agent will receive before you install.
Tier: POWERFUL · Category: Engineering Quality Assurance · Dependencies: None (Python stdlib only)
Meta-skill that validates, tests, and scores skills in this repository. Four tools, run from the repo root with full paths:
scripts/skill_validator.py — structure + documentation compliancescripts/script_tester.py — Python script syntax/imports/runtime/output testingscripts/quality_scorer.py — multi-dimensional scoring with letter gradescripts/security_scorer.py — security posture scoring (also available via quality_scorer.py --include-security)Scope note: this skill's tier line-count minimums measure legacy skills. For authoring new skills,
engineering/write-a-skill(SKILL.md under ~100 lines, Matt Pocock doctrine) is the binding standard — do not pad a new skill to satisfy a tier minimum here.
# 1. Validate structure (exit non-zero on failure — usable as a gate)
python3 engineering/skills/skill-tester/scripts/skill_validator.py engineering/skills/self-eval --json
# 2. Test the skill's Python scripts (30s default timeout per script)
python3 engineering/skills/skill-tester/scripts/script_tester.py engineering/skills/self-eval --json
# 3. Score quality (fail CI below threshold with --minimum-score)
python3 engineering/skills/skill-tester/scripts/quality_scorer.py engineering/skills/self-eval --json --detailed --minimum-score 75
Consume the JSON: validator emits overall_score, compliance_level, per-check checks{}; scorer emits overall_score, letter_grade, tier_recommendation, dimensions, and an improvement_roadmap — work the roadmap top-down, then re-run until the target score is met.
For repo-wide auditing prefer scripts/audit_skills.py at the repo root (wraps the write-a-skill checklist runner across all skills).
--tier BASIC|STANDARD|POWERFUL)--timeout, default 30s)--help functionality verification; sample-data runs compared against expected_outputs/Four dimensions, 25% each: Documentation (depth, examples, references), Code Quality (complexity, error handling, output consistency), Completeness (required dirs, sample data, expected outputs), Usability (help text, example clarity). Outputs 0-100 + A-F grade + tier recommendation.
| Tier | SKILL.md | Scripts | CLI surface |
|---|---|---|---|
| BASIC | ≥ 100 lines | 1 (100-300 LOC) | basic argparse |
| STANDARD | ≥ 200 lines | 1-2 (300-500 LOC) | subcommands, JSON + text output |
| POWERFUL | ≥ 300 lines | 2-3 (500-800 LOC) | multiple modes, CI integration |
(Advisory for legacy skills; new skills follow write-a-skill — see scope note above.)
# GitHub Actions: gate changed skills
- name: "validate-changed-skills"
run: |
for skill in $changed_skills; do
python3 engineering/skills/skill-tester/scripts/skill_validator.py "$skill" --json
python3 engineering/skills/skill-tester/scripts/script_tester.py "$skill"
python3 engineering/skills/skill-tester/scripts/quality_scorer.py "$skill" --minimum-score 75
done
Pre-commit hook: run the validator on the staged skill directory and block the commit on non-zero exit.
A skill "passes" when, in one run from repo root:
skill_validator.py <skill> --json exits 0,script_tester.py <skill> reports all scripts passing, andquality_scorer.py <skill> --minimum-score <target> exits 0.If any step fails, apply the top improvement_roadmap item and re-run all three — never report a partial pass.
--timeout or optimize the script under testReferences: references/ holds the structure specification, tier requirements matrix, and scoring rubric the tools implement.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Audit web applications for accessibility issues against WCAG 2.2 Level A and AA, the web accessibility standards. Get framework-specific code fixes, contrast checks, verification steps, and reports, with options for automated checks in development pipelines.
Plan A/B tests that compare page variations, from headlines to pricing layouts. Define a hypothesis, choose success metrics, calculate sample size and duration, and prepare launch checks and results reports to support decisions.
Create paid ad headlines and body copy tailored to each platform's character limits. Use existing performance data to develop variations, explore new messaging angles, and check copy for length, quality, and potential rejection risks.
Review code changes from three critical perspectives: production failures, maintainability, and security. Get a consolidated report that ranks findings by severity and provides merge guidance, helping examine assumptions that an ordinary self-review may overlook.
Audit and improve content intended for citation in AI answers. Review evidence, author information, and structure, generate revised content and structured metadata, and keep a local record of observed citations across AI services.
Create signed records for consequential agent actions such as deployments, deletions, and payments. Decide which actions need evidence, record the agent, action, target, and policy, then verify the resulting certificate offline to detect later changes.