Skip to content
cccskills
FreePublished on GitHubEnglish overview

skill-comply

Check whether coding agents follow their assigned skills and rules across three levels of prompt support. Generate test scenarios, inspect tool call sequences, and get reports showing compliance scores and the order of observed actions.

OriginalShow the original description

Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.

AI rating80/ 100See how to install

Good for

  • Checking whether a newly added rule is followed
  • Testing compliance without explicit reminders
  • Reviewing tool call order and missed steps
  • Generating scenarios without running agents

Overview

What it does

This skill checks whether an AI agent follows its assigned skills or rules. It derives an expected sequence of actions from Markdown and generates scenarios at three levels: instructions that support the rule, neutral instructions, and competing instructions. An LLM classifies tool calls made during execution, and the skill checks their order. Reports show compliance scores and timelines for each scenario.

When it helps

Use it after adding a rule, or when you want to know whether a procedure is followed without an explicit reminder. Reports include the expected steps and the actual prompts, so you can see which stages were missed. For steps with low compliance, it also suggests moving the action to a hook that starts it automatically.

Example requests

  • “Check whether this testing.md is actually being followed.”
  • “Generate verification scenarios from SKILL.md without running them.”

Notes

The execution examples use uv, Python, and claude -p. Normal verification calls models and incurs costs. With --dry-run, only specifications and scenarios are generated; the agent does not run. Verification of agent definitions only checks whether the agent is invoked. Checking its internal workflow is not yet supported.

This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.

Objective evaluation

AI rated this skill on four points using only the contents of its SKILL.md. Stars and popularity are not taken into account.

Clear instructionsIs it clear when to use it and what it does?
AGoodIt explicitly describes checking skill and rule compliance through scenarios at three prompt-support levels.
Concrete stepsAre steps, inputs, outputs, and examples spelled out?
AGoodIt lists six stages from specification generation to reporting, execution commands, and four report components.
SafetyAre sending data, deleting files, and running commands disclosed?
BPartly lackingExecution through claude -p and trace capture are described, but execution permissions and isolation boundaries are not specified.
Easy setupDoes it need few tools or API keys, and are they documented?
BPartly lackingExamples use uv, Python, and claude -p, but installation and authentication procedures are not described.

The total counts each point as A = 25, B = 15, C = 5. Rated by AI (OpenAI Codex) as of Oct 19, 2026. Treat it as a guide and read the original before installing.

Disagree? Request a re-evaluation

Tell us which point is wrong and why. No login needed. We will check it against the SKILL.md, redo the evaluation, and show the result here.

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

Files included (22)

  • SKILL.md2.4 KB
  • fixtures/compliant_trace.jsonl1.0 KB
  • fixtures/noncompliant_trace.jsonl629 B
  • fixtures/tdd_spec.yaml1.2 KB
  • prompts/classifier.md1.0 KB
  • prompts/scenario_generator.md1.9 KB
  • prompts/spec_generator.md1.6 KB
  • pyproject.toml360 B
  • scripts/__init__.py0 B
  • scripts/classifier.py2.7 KB
  • scripts/grader.py7.6 KB
  • scripts/parser.py2.7 KB
  • scripts/report.py6.3 KB
  • scripts/run.py4.2 KB
  • scripts/runner.py9.9 KB
  • scripts/scenario_generator.py1.8 KB
  • scripts/spec_generator.py2.2 KB
  • scripts/utils.py411 B
  • tests/test_classifier.py3.1 KB
  • tests/test_grader.py17.7 KB
  • tests/test_parser.py3.1 KB
  • tests/test_runner.py21.8 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

skill-comply: Automated Compliance Measurement

Measures whether coding agents actually follow skills, rules, or agent definitions by:

  1. Auto-generating expected behavioral sequences (specs) from any .md file
  2. Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
  3. Running claude -p and capturing tool call traces via stream-json
  4. Classifying tool calls against spec steps using LLM (not regex)
  5. Checking temporal ordering deterministically
  6. Generating self-contained reports with spec, prompts, and timelines

Supported Targets

  • Skills (skills/*/SKILL.md): Workflow skills like search-first, TDD guides
  • Rules (rules/common/*.md): Mandatory rules like testing.md, security.md, git-workflow.md
  • Agent definitions (agents/*.md): Whether an agent gets invoked when expected (internal workflow verification not yet supported)

When to Activate

  • User runs /skill-comply <path>
  • User asks "is this rule actually being followed?"
  • After adding new rules/skills, to verify agent compliance
  • Periodically as part of quality maintenance

Usage

# Full run
uv run python -m scripts.run ~/.claude/rules/common/testing.md

# Dry run (no cost, spec + scenarios only)
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md

# Custom models
uv run python -m scripts.run --gen-model haiku --model sonnet <path>

Key Concept: Prompt Independence

Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.

Report Contents

Reports are self-contained and include:

  1. Expected behavioral sequence (auto-generated spec)
  2. Scenario prompts (what was asked at each strictness level)
  3. Compliance scores per scenario
  4. Tool call timelines with LLM classification labels

Advanced (optional)

For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

accessibility

FreeSummary

Design and audit interfaces for Web, iOS, and Android using WCAG 2.2 accessibility guidance. Check screen-reader labels, keyboard navigation, contrast, and control sizes to identify barriers and guide implementation for people using assistive technology.

  • Adding descriptive labels to icon-only buttons
  • Reviewing keyboard navigation and modal focus
  • Checking contrast and control sizes
affaan-m/ECC283.5KUpdated Oct 16, 2026

Investigate unreliable AI agents across 12 layers, including instructions, memory, tool execution, and display. Use code and logs to trace failures, rank findings by severity, and produce an ordered repair plan that prioritizes code changes.

  • Investigating failures that appear only inside an agent application
  • Tracing old conversation content leaking into new sessions
  • Checking skipped or falsely claimed tool execution
affaan-m/ECC283.5KUpdated Oct 16, 2026

agent-eval

FreeSummaryAI rating 80

Compare coding agents on tasks from your own codebase. Define success checks, run agents in separate Git working directories, and report pass rates, available API costs, completion time, and consistency across repeated trials.

  • Comparing coding agents on real development tasks
  • Evaluating a new tool or model before adoption
  • Checking performance after agent updates
affaan-m/ECC283.5KUpdated Oct 16, 2026

Design and refine an AI agent’s available actions, tool inputs, and result formats. Define recovery steps for errors, manage the information kept in context, and evaluate changes through completion rates, retries, and cost per successful task.

  • Define an AI agent’s tool set and input formats
  • Make tool results and next steps easier to interpret
  • Specify safe retries and stopping conditions
affaan-m/ECC283.5KUpdated Oct 16, 2026

Diagnose AI agent runs that fail repeatedly, loop through tools, or lose track of the task. Capture errors and environment details, test a focused recovery action, and produce a report explaining the evidence and outcome.

  • Diagnosing repeated failures in an agent run
  • Investigating loops through the same tool calls
  • Reviewing output degradation as context grows
affaan-m/ECC283.5KUpdated Oct 5, 2026

agent-introspection-debugging

FreeSummaryAI rating 100

Diagnose failed AI agent runs by capturing errors, checking likely causes, and choosing small, reversible recovery actions. The workflow produces a structured report of the failure, recovery steps, and evidence, helping explain repeated retries or stalled progress.

  • Diagnosing repeated tool calls with no progress
  • Investigating quality loss as context grows
  • Checking file paths and environment mismatches
affaan-m/ECC283.5KUpdated Oct 16, 2026

See all skills by affaan-m

Report a problem with this skill