跳到正文
cccskills
免费在 GitHub 公开

property-based-testing

Writes, reviews, and debugs property-based tests, which generate inputs to check general rules. Helps test conversions, parsers, normalization, and state invariants, assess whether existing assertions are meaningful, and distinguish faulty rules from implementation bugs.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

Writes, reviews, and debugs property-based tests — Hypothesis, fast-check, proptest, jqwik, rapid, and Echidna or Medusa for Solidity invariants. Use whenever tests should cover a whole input domain instead of a hand-picked list of examples: encode/decode and serialize/deserialize pairs, parsers, canonicalizers and normalizers, validators, numeric and Decimal types, comparators and sort order, data structures, and smart-contract state invariants. Also use when adding cases to an existing @given, fast-check, or proptest suite, when judging whether existing property tests assert anything real, and when a generator has shrunk a counterexample and you need to tell a wrong property from a genuine bug. Not for coverage-guided binary fuzzing (libFuzzer, AFL), mutation-testing campaigns, static analysis, benchmarking, or end-to-end UI tests.

查看安装方法

适用场景

  • Testing encode/decode round trips across generated inputs
  • Checking normalization and sorting properties
  • Reviewing whether existing property tests assert meaningful rules
  • Separating incorrect properties from implementation bugs
  • Testing smart contract state invariants

包含的文件(9)

  • SKILL.md4.2 KB
  • agents/openai.yaml254 B
  • assets/trail-of-bits-mark.svg3.0 KB
  • README.md15.0 KB
  • references/generating.md2.6 KB
  • references/interpreting-failures.md3.3 KB
  • references/libraries.md2.9 KB
  • references/refactoring.md4.7 KB
  • references/reviewing.md2.8 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Property-Based Testing

An example test asserts one point. A property asserts a rule over the whole input domain and lets the generator hunt for the counterexample. That trade is worth making when the code has an algebraic shape — an inverse, an invariant, an oracle — and not otherwise. Code with no such shape gets example tests; saying so is a valid outcome.

Check first whether the shape is missing or merely buried. A calculation wrapped in I/O, a string built by concatenation, an in-place mutation — each has a property and no seam to assert it through. See references/refactoring.md before concluding there is nothing to assert.

Property catalog

PropertyFormulaWhere it applies
Roundtripdecode(encode(x)) == xSerialization, conversion pairs
Inversef(g(x)) == xencrypt/decrypt, compress/decompress
Oraclenew(x) == reference(x)Optimization, refactoring, reimplementation
Idempotencef(f(x)) == f(x)Normalization, formatting, sorting
InvariantHolds before and afterAny transformation, contract state
Easy to verifyis_sorted(sort(x))Complex algorithms with cheap checkers
Commutativityf(a, b) == f(b, a)Binary and set operations
Associativityf(f(a,b), c) == f(a, f(b,c))Combining operations
Identityf(x, e) == xOperations with a neutral element

Strength ordering, weakest to strongest: no crash → type preservation → invariant → idempotence → roundtrip / oracle.

Assert the strongest property the code supports. "No crash" alone rarely justifies the dependency — if that is all you can find, either a small rearrangement exposes something stronger, or the honest report is that this code is a poor PBT candidate. Rule out the first before settling for the second.

The two ways a property test asserts nothing

  • Tautology. assert add(a, b) == a + b restates the implementation; no bug they share can fail it. Pick a property that constrains the function without recomputing it. Note the exception: f(x) == f(x) is a genuine determinism property when f is not obviously pure — serializers over dicts or sets, hashing, anything reading the clock.
  • Vacuity. assume() that filters out nearly every input passes without exercising anything, and self-contradictory assume() passes having run zero cases. Push constraints into the strategy so the generator produces valid inputs directly.

Where to look next

Load the one that matches the task in front of you:

TaskFile
Writing new tests, designing strategiesreferences/generating.md
The code has no property to assert yetreferences/refactoring.md
Reviewing existing property testsreferences/reviewing.md
A property test just failedreferences/interpreting-failures.md
Library choice, Echidna and Medusareferences/libraries.md

Introducing PBT to a project that lacks it

If the project already uses a PBT library, just write the tests in it. If it does not, adding one is a dependency decision that belongs to the user — offer it once with the specific property you would write, and take the answer either way.

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Build and test code with AddressSanitizer, a tool that detects invalid memory access during execution. Configure checks for C/C++ and unsafe Rust, investigate crash reports, and adjust settings for fuzzing, or testing with generated inputs.

中文简介正在准备中。当前显示英语简介。

  • Detect memory errors while fuzzing C/C++
  • Test unsafe Rust and calls to external libraries
  • Interpret an ASan crash report
trailofbits/skills76402026年10月16日 更新

aflpp

免费

Sets up AFL++ to test C/C++ programs with varied generated inputs, a technique called fuzzing. Covers running tests across CPU cores, monitoring progress, reducing input collections, and reproducing crashes for investigation.

中文简介正在准备中。当前显示英语简介。

  • Running C/C++ fuzzing across multiple CPU cores
  • Exploring further after libFuzzer progress stalls
  • Monitoring fuzzing progress and stability
trailofbits/skills76402026年10月16日 更新

Audits GitHub Actions workflows that run AI coding agents. Traces how attacker-controlled content reaches prompts, checks permissions and sandbox settings, and reports evidence, severity, and fixes for prompt injection and related configuration risks.

中文简介正在准备中。当前显示英语简介。

  • Reviewing workflows that run AI coding agents
  • Tracing issue and PR content into agent prompts
  • Checking sandbox settings and user allowlists
trailofbits/skills76402026年10月16日 更新

Reviews Algorand smart contracts written in TEAL or PyTeal for 11 vulnerability patterns. Produces a complete coverage table and findings with code references, severity ratings, and suggested fixes, with optional automated checks using Tealer.

中文简介正在准备中。当前显示英语简介。

  • Security checks before an Algorand contract audit
  • Reviewing TEAL or PyTeal transaction validation
  • Verifying fixes for reported vulnerabilities
trailofbits/skills76402026年10月16日 更新

atheris

免费

Set up Atheris to test Python code and C extensions with automatically varied inputs. It guides test harness creation, coverage tracking to explore more code, and AddressSanitizer setup for finding memory errors.

中文简介正在准备中。当前显示英语简介。

  • Fuzzing a Python package or parser
  • Investigating memory corruption in Python C extensions
  • Turning raw bytes into typed test arguments
trailofbits/skills76402026年10月16日 更新

Connect static analysis results and human audit notes to a Trailmark code graph. Group findings by severity and compare them with input flow, impact, and privilege boundaries to prioritize investigation and understand findings in their code context.

中文简介正在准备中。当前显示英语简介。

  • Connecting Semgrep or CodeQL results to code structure
  • Combining human audit notes with automated findings
  • Prioritizing findings using input flow and impact
trailofbits/skills76402026年10月16日 更新

查看 trailofbits 的全部技能

报告此技能的问题