본문으로 건너뛰기
cccskills
무료GitHub에 공개

skill-comply

Check whether coding agents follow their assigned skills and rules across three levels of prompt support. Generate test scenarios, inspect tool call sequences, and get reports showing compliance scores and the order of observed actions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

원문원문 설명 보기

Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.

설치 방법 보기

이럴 때 유용합니다

  • Checking whether a newly added rule is followed
  • Testing compliance without explicit reminders
  • Reviewing tool call order and missed steps
  • Generating scenarios without running agents

포함된 파일(22)

  • SKILL.md2.4 KB
  • fixtures/compliant_trace.jsonl1.0 KB
  • fixtures/noncompliant_trace.jsonl629 B
  • fixtures/tdd_spec.yaml1.2 KB
  • prompts/classifier.md1.0 KB
  • prompts/scenario_generator.md1.9 KB
  • prompts/spec_generator.md1.6 KB
  • pyproject.toml360 B
  • scripts/__init__.py0 B
  • scripts/classifier.py2.7 KB
  • scripts/grader.py7.6 KB
  • scripts/parser.py2.7 KB
  • scripts/report.py6.3 KB
  • scripts/run.py4.2 KB
  • scripts/runner.py9.9 KB
  • scripts/scenario_generator.py1.8 KB
  • scripts/spec_generator.py2.2 KB
  • scripts/utils.py411 B
  • tests/test_classifier.py3.1 KB
  • tests/test_grader.py17.7 KB
  • tests/test_parser.py3.1 KB
  • tests/test_runner.py21.8 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

skill-comply: Automated Compliance Measurement

Measures whether coding agents actually follow skills, rules, or agent definitions by:

  1. Auto-generating expected behavioral sequences (specs) from any .md file
  2. Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing)
  3. Running claude -p and capturing tool call traces via stream-json
  4. Classifying tool calls against spec steps using LLM (not regex)
  5. Checking temporal ordering deterministically
  6. Generating self-contained reports with spec, prompts, and timelines

Supported Targets

  • Skills (skills/*/SKILL.md): Workflow skills like search-first, TDD guides
  • Rules (rules/common/*.md): Mandatory rules like testing.md, security.md, git-workflow.md
  • Agent definitions (agents/*.md): Whether an agent gets invoked when expected (internal workflow verification not yet supported)

When to Activate

  • User runs /skill-comply <path>
  • User asks "is this rule actually being followed?"
  • After adding new rules/skills, to verify agent compliance
  • Periodically as part of quality maintenance

Usage

# Full run
uv run python -m scripts.run ~/.claude/rules/common/testing.md

# Dry run (no cost, spec + scenarios only)
uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md

# Custom models
uv run python -m scripts.run --gen-model haiku --model sonnet <path>

Key Concept: Prompt Independence

Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it.

Report Contents

Reports are self-contained and include:

  1. Expected behavioral sequence (auto-generated spec)
  2. Scenario prompts (what was asked at each strictness level)
  3. Compliance scores per scenario
  4. Tool call timelines with LLM classification labels

Advanced (optional)

For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Design and audit interfaces for Web, iOS, and Android using WCAG 2.2 accessibility guidance. Check screen-reader labels, keyboard navigation, contrast, and control sizes to identify barriers and guide implementation for people using assistive technology.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Adding descriptive labels to icon-only buttons
  • Reviewing keyboard navigation and modal focus
  • Checking contrast and control sizes
affaan-m/ECC28.3만2026년 10월 16일 업데이트

Investigate unreliable AI agents across 12 layers, including instructions, memory, tool execution, and display. Use code and logs to trace failures, rank findings by severity, and produce an ordered repair plan that prioritizes code changes.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Investigating failures that appear only inside an agent application
  • Tracing old conversation content leaking into new sessions
  • Checking skipped or falsely claimed tool execution
affaan-m/ECC28.3만2026년 10월 16일 업데이트

Compare coding agents on tasks from your own codebase. Define success checks, run agents in separate Git working directories, and report pass rates, available API costs, completion time, and consistency across repeated trials.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Comparing coding agents on real development tasks
  • Evaluating a new tool or model before adoption
  • Checking performance after agent updates
affaan-m/ECC28.3만2026년 10월 16일 업데이트

Design and refine an AI agent’s available actions, tool inputs, and result formats. Define recovery steps for errors, manage the information kept in context, and evaluate changes through completion rates, retries, and cost per successful task.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Define an AI agent’s tool set and input formats
  • Make tool results and next steps easier to interpret
  • Specify safe retries and stopping conditions
affaan-m/ECC28.3만2026년 10월 16일 업데이트

Diagnose failed AI agent runs by capturing errors, checking likely causes, and choosing small, reversible recovery actions. The workflow produces a structured report of the failure, recovery steps, and evidence, helping explain repeated retries or stalled progress.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Diagnosing repeated tool calls with no progress
  • Investigating quality loss as context grows
  • Checking file paths and environment mismatches
affaan-m/ECC28.3만2026년 10월 16일 업데이트

Diagnose AI agent runs that fail repeatedly, loop through tools, or lose track of the task. Capture errors and environment details, test a focused recovery action, and produce a report explaining the evidence and outcome.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Diagnosing repeated failures in an agent run
  • Investigating loops through the same tool calls
  • Reviewing output degradation as context grows
affaan-m/ECC28.3만2026년 10월 5일 업데이트

affaan-m의 스킬 모두 보기

이 스킬의 문제 신고