Perf regression test

The deciding step is whether to run this at all. Running a two-minute benchmark on a docs PR trains people to skip it entirely; running it on an unquiesced, contended machine produces a "regression" that is really just noise, and one false alarm burns more trust than ten skipped checks would have saved. Read this whole page before running anything.

Never run any of this in CI. GitHub-hosted runners are shared VMs whose run-to-run spread (2-3x) exceeds any regression worth catching — a threshold loose enough not to false-positive there catches nothing. See .claude/notes/perf-tools-not-in-ci.md.

1. Decide whether to run it

Run only when the diff touches:

src/elements/          include/elements/
src/simulation/        include/simulation/
src/tools/             include/tools/math.h
include/tools/fft_convolution.h
include/tools/simd_dispatch.h

Skip for wiki/, examples/, src/user_interface/, src/visualization/, tests/-only and docs-only diffs. Say that it was skipped and why — a silent skip reads as a pass to anyone reading the PR later.

Also skip (for now, re-run later) when:

• Other builds or agents are active on this machine. Wall-clock measurement under concurrent load is worthless — it will not just be noisy, it will be systematically biased toward "everything got slower." Wait for the machine to go quiet, or say clearly that the check could not be run cleanly.
• No baseline exists yet for this environment. --check will tell you this itself (exit code 2) — see step 3.

2. Build

cmake --build --preset release --target dnf_composer_deckbench --parallel 4

Use the build-and-test skill for the general build workflow. Let the build finish before timing anything — a build running in the background will contend for CPU exactly like a second agent would.

3. Run under the hygiene wrapper

scripts\bench.ps1 build\release\tests\dnf_composer_deckbench.exe --check

scripts/bench.sh build/release/tests/dnf_composer_deckbench.exe --check

The wrapper pins CPU affinity, raises priority, and fixes clocks where the platform allows it, then records what it actually managed to do into the tool's own JSON output — see .claude/performance-workplan/WP-07-machine-hygiene-scripts.md for why. Running the tool directly (unwrapped) works, but expect a wider noise floor and more [NOISY] / exit-3 results.

4. Read the exit code

Code	Meaning	What to do
0	Every deck within threshold	Report the deltas anyway — they're useful even when nothing regressed
1	At least one deck regressed	Go to step 5 to attribute it, then decide: real regression, or does the change justify it?
2	No baseline for this fingerprint, or a deck's file hash no longer matches the baseline's	Do not compare. Report the check as unverified. --record only ever happens on a known-good tree — see step 6
3	Too noisy to conclude	Inconclusive, not a pass and not a failure. Re-run under the wrapper, or on a quieter machine, or report it as unverified

--check's own table (deck / baseline / current / delta / verdict) prints on every exit code — read it, don't just act on the number.

5. Attribute a regression to an element

build\release\tests\dnf_composer_profiler.exe --deck <the deck that regressed>

Find the deck's path in tests/benchmark/decks.json from the tier name --check printed (small / medium / large-a / large-b). The profiler drives each element's own step() directly — no library instrumentation involved — and prints (and can write as JSON via --json) each element's mean µs and share of the step. Compare against the matching session in tests/profiler/profile.md to see which element's share grew.

Then check the numerics didn't move too. Every deck in decks.json is also a validation fixture with a committed reference CSV (see tests/benchmark/DECKS.md) — run the matching case in dnf_composer_tests (GoldenFieldDynamics* / SpectralGolden2D* depending on which deck). A change that is faster and numerically different is not an optimization; it's a different, unverified algorithm.

6. Updating a baseline

scripts\bench.ps1 build\release\tests\dnf_composer_deckbench.exe --record --force

Only on a tree you already know is correct, and only deliberately — the same rule tests/golden/README.md sets for golden data. State why the number moved in the PR. A baseline quietly re-recorded to turn a red check green is worse than no baseline at all: it launders a real regression into the new normal.

--record refuses on its own if the run is noisy, and refuses to overwrite an existing baseline without --force — both are the tool protecting itself from exactly this failure mode, not obstacles to route around.

What this cannot tell you

This gate catches regressions in the four architectures decks.json covers (1D direct, a 2-term 2D kernel, and the direct/spectral pair straddling kFFTTapThreshold) at the sizes those decks use. It says nothing about an architecture or grid size outside that set. If the change is architecturally novel, note that in the PR rather than treating a clean --check as full coverage.