본문으로 건너뛰기
cccskills
무료GitHub에 공개

mutation-testing

Assess test strength by introducing deliberate code changes with mewt or muton. Configure test runs, examine changes that tests miss, distinguish harmless equivalents from testing gaps, and investigate potential bugs in the affected code.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

원문원문 설명 보기

Configures mewt or muton campaigns, analyzes surviving mutants, and investigates bugs exposed by testing gaps. Use when setting up mutation testing, reviewing campaign results, identifying equivalent mutants, or finding bugs from surviving mutations.

설치 방법 보기

이럴 때 유용합니다

  • Set up and scope a mutation testing run
  • Investigate code changes that tests fail to detect
  • Distinguish equivalent changes from testing gaps
  • Prepare a formal report of mutation testing results

포함된 파일(12)

  • SKILL.md6.1 KB
  • agents/openai.yaml243 B
  • assets/trail-of-bits-mark.svg3.0 KB
  • references/blockchain-patterns.md4.8 KB
  • references/equivalent-mutants.md7.0 KB
  • references/input-formats.md2.4 KB
  • references/optimization-strategies.md8.7 KB
  • references/report-template.md4.3 KB
  • references/severity-classification.md5.9 KB
  • workflows/analyzing-results.md6.7 KB
  • workflows/bug-hunter.md7.4 KB
  • workflows/configuration.md11.3 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

Mutation Testing (mewt/muton)

Routes to the right mutation testing workflow and loads the references that workflow needs.

Note: muton and mewt share identical interfaces. Examples use mewt; substitute muton and its file names (muton.toml, muton.sqlite) for muton projects.

mewt --help and mewt <subcommand> --help are the source of truth for command-line behavior. Examples below reflect the mewt 4.x API; run --help when a flag looks unfamiliar or a command fails.

When to Use

Use this skill when the user:

  • Mentions "mewt", "muton", or "mutation testing"
  • Wants to configure, scope, or speed up a mutation testing campaign
  • Wants to analyze mutation results — surviving/uncaught mutants, equivalent mutants, kill rate
  • Wants to use mutation results to find bugs in the source code

When NOT to Use

Do not use this skill when the user asks about tests or line coverage without any mutation testing context.


Routing

Pick the workflow, then load it together with the references listed for it. Workflows and references do not load each other — that decision belongs here.

Setting up, scoping, or speeding up a campaign → workflows/configuration.md → Also load references/optimization-strategies.md when the campaign estimate is long enough to need trimming, or the user asks to make it faster.

Campaign finished, hunting for bugs in untested code → workflows/bug-hunter.md

Turning results into a formal analysis report → workflows/analyzing-results.md, plus:

Anything else → run mewt --help or mewt <subcommand> --help, then assist directly.


Essential Commands

# Set up and run
mewt init                    # Create config and database
mewt mutate [paths]          # Generate mutants without testing them
mewt run [paths]             # Generate mutants and run the campaign

# Read results
mewt status                  # Overview with per-file breakdown
mewt results                 # Uncaught mutants (default view)
mewt results --all           # Every outcome, not just uncaught
mewt results --format json   # json | sarif | ids | table

# Narrow down (these filters work on both `results` and `print mutants`)
mewt results --target 'src/auth/**'   # Quote globs so the shell does not expand them
mewt results --severity high,medium
mewt results --mutation-types ER,CR
mewt results --status Uncaught        # Uncaught | TestFail | Skipped | Timeout
mewt results --line 42

# Investigate and re-test
mewt print mutant --id [id]              # View the mutated code
mewt test --ids [ids]                    # Re-test specific mutants
mewt test --ids-file uncaught_ids.txt    # Re-test IDs from a file, or '-' for stdin

# Inspect configuration
mewt print config                        # Effective config
mewt print targets                       # Files actually mutated
mewt print mutations --language [lang]   # Mutations and severities for a language

Language labels are canonical family or family/dialect values in mewt 4.x — for example rust, javascript/ts, move/sui, move/iota.


What Results Mean

  • Caught/TestFail: tests detected the mutation (good)
  • Uncaught: tests did not detect the change. Inspect the code to distinguish a testing gap from an equivalent mutation.
  • Timeout: tests took too long — inconclusive, not evidence of coverage
  • Skipped: a less severe mutant was skipped because a more severe mutant on the same line was uncaught

Interpreting Mutation Types

mewt print mutations --language [lang] lists every mutation slug, description, and severity for a language, and is authoritative — the operator set grows with each release. What that output does not tell you is what a survivor means, which is where prioritization comes from:

SeverityRepresentative slugsWhat an uncaught mutant tells you
HighER (Error Replacement)Tests tolerate the injected error. Investigate whether the path executes, whether error handling masks the change, and whether assertions check the outcome.
MediumCR (Comment Replacement)Removing the statement does not fail the tests. Check whether its effects matter and whether assertions observe them.
MediumIF/IT (If False/True), NR (Negation Removal)Tests do not distinguish the changed condition. Both constant replacements surviving can indicate an unexecuted condition or weak assertions on the branch outcomes.
LowOperator shuffles (AOS, COS, LOS, BOS, shift/assignment variants), BL, AS, LC, WFCheck boundary inputs, arithmetic assertions, and semantic equivalence. The mutation result alone does not establish whether the code executed.

Severity ranks the mutation, not the risk. A low-severity survivor in a fee calculation matters more than a high-severity survivor in a log line — weigh what the mutated code does. Filter with --severity to work through the results in priority order.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Build and test code with AddressSanitizer, a tool that detects invalid memory access during execution. Configure checks for C/C++ and unsafe Rust, investigate crash reports, and adjust settings for fuzzing, or testing with generated inputs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Detect memory errors while fuzzing C/C++
  • Test unsafe Rust and calls to external libraries
  • Interpret an ASan crash report
trailofbits/skills7.6천2026년 10월 16일 업데이트

aflpp

무료

Sets up AFL++ to test C/C++ programs with varied generated inputs, a technique called fuzzing. Covers running tests across CPU cores, monitoring progress, reducing input collections, and reproducing crashes for investigation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Running C/C++ fuzzing across multiple CPU cores
  • Exploring further after libFuzzer progress stalls
  • Monitoring fuzzing progress and stability
trailofbits/skills7.6천2026년 10월 16일 업데이트

Audits GitHub Actions workflows that run AI coding agents. Traces how attacker-controlled content reaches prompts, checks permissions and sandbox settings, and reports evidence, severity, and fixes for prompt injection and related configuration risks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Reviewing workflows that run AI coding agents
  • Tracing issue and PR content into agent prompts
  • Checking sandbox settings and user allowlists
trailofbits/skills7.6천2026년 10월 16일 업데이트

Reviews Algorand smart contracts written in TEAL or PyTeal for 11 vulnerability patterns. Produces a complete coverage table and findings with code references, severity ratings, and suggested fixes, with optional automated checks using Tealer.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Security checks before an Algorand contract audit
  • Reviewing TEAL or PyTeal transaction validation
  • Verifying fixes for reported vulnerabilities
trailofbits/skills7.6천2026년 10월 16일 업데이트

atheris

무료

Set up Atheris to test Python code and C extensions with automatically varied inputs. It guides test harness creation, coverage tracking to explore more code, and AddressSanitizer setup for finding memory errors.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Fuzzing a Python package or parser
  • Investigating memory corruption in Python C extensions
  • Turning raw bytes into typed test arguments
trailofbits/skills7.6천2026년 10월 16일 업데이트

Connect static analysis results and human audit notes to a Trailmark code graph. Group findings by severity and compare them with input flow, impact, and privilege boundaries to prioritize investigation and understand findings in their code context.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

  • Connecting Semgrep or CodeQL results to code structure
  • Combining human audit notes with automated findings
  • Prioritizing findings using input flow and impact
trailofbits/skills7.6천2026년 10월 16일 업데이트

trailofbits의 스킬 모두 보기

이 스킬의 문제 신고