본문으로 건너뛰기
cccskills
무료GitHub에 공개

absolute-deflake

Flaky test fixes: detect nondeterministic tests empirically (repeat/shuffle/parallel runs), diagnose the root cause, fix it — never retry/skip/sleep — and verify across many randomized runs. Triggers on "absolute deflake", "fix flaky tests", "CI is flaky", "this test fails randomly/intermittently".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

설치 방법 보기

포함된 파일(3)

  • SKILL.md4.9 KB
  • README.md654 B
  • references/health-engine.md5.3 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

Start your first response with the 🧪 emoji.

Absolute Deflake

Find tests that pass and fail nondeterministically, diagnose the root cause of each, and fix it — not by retrying or skipping, but by removing the source of nondeterminism. Output is evidence (failure rate per test) → cause → fix, verified by repeated runs.

Runs the shared engine in references/health-engine.md — read it for the DETECT → SCAN → TRIAGE → FIX → VERIFY → REPORT loop and the safety contract. This file covers only what's specific to flaky tests.


When to use

  • "Our CI is flaky", "this test fails randomly", "fix the intermittent failures".
  • A test passes locally but fails in CI (or vice versa), or fails ~1 in N runs.
  • Burning down a backlog of retry/skip-marked tests that mask real flakiness.

Not for tests that fail deterministically — that's a real bug or a real regression (/absolute work for a fix, or just fix it). deflake targets nondeterministic failures.


What it scans

Establish flakiness empirically — a test isn't flaky because someone said so. Use preferences.health.deflakeRuns from config as the default N for repeat-runs (else 20):

EcosystemRepeat-run / detect
Jest/Vitestrun suite N× (--run loop), randomize order (--shuffle / testSequencer)
pytestpytest-randomly + pytest --count=N (pytest-repeat); -p no:randomly to A/B
Gogo test -count=N -shuffle=on ./..., -race

Also mine signals: existing retry/flaky/skip annotations, CI history if reachable, and run the suite both in isolation and in full/parallel — order- and concurrency- dependent failures only show one way. Record a failure rate per suspect test.


Common root causes (diagnose, don't guess)

CauseTellFix
Test-order / shared statepasses alone, fails in suite (or vice versa)isolate state; reset/teardown between tests
Time / clockfails near midnight, DST, or under loadfake timers / inject clock; no real sleep
Async race / missing awaitfails under parallelism or slow CIawait the actual condition; no fixed timeouts
Randomnessfails ~X% with no patternseed the RNG; fix the seed in tests
Network / external I/Ofails offline or on slow linksmock/stub the boundary
Unordered collectionsfails on map/set iteration ordersort before asserting
Resource leak / port reusefails on repeat or parallel runsunique resources; clean up

Risk ranking (TRIAGE)

WaveClassDefault
1clear, isolated cause (seed, await, fake clock, sort)fix now
2shared-state / ordering — needs fixture refactorfix this pass, per test
3flakiness pointing at a real product race, not just the testgated — surface; may be a genuine bug to fix in code

A flaky test sometimes means the code has a race, not the test. Don't "stabilize" the test into hiding a real concurrency bug — flag wave-3 cases for a real fix.


Fix & verify

  • Fix the cause. Then prove it: re-run the test many times (and shuffled / parallel / with -race) — green once is not deflaked; green across N randomized runs is.
  • Remove the retry/skip/flaky annotation that was masking it once the cause is fixed.
  • Never "fix" by adding retries, raising timeouts blindly, sleep, or skipping the test — that hides flakiness, doesn't remove it.
  • Re-run the full suite to confirm the fix didn't destabilize neighbors.

Gotchas

  1. Retry/skip as a fix. Masks the flake, ships the nondeterminism. Forbidden here.
  2. sleep to dodge a race. Slows the suite and still flakes under load. Await the condition.
  3. One green run = done. Flakes are probabilistic — verify with many randomized runs.
  4. Stabilizing a real product race. If the code races, fix the code, not just the assertion.
  5. Ignoring order/parallel dimension. Run isolated and in-suite; the bug hides in whichever you skip.

Companion commands

  • /absolute upgrade — a flaky suite makes upgrade verification unreliable; deflake first.
  • /absolute debt — flaky-test annotations are test debt; this clears them at the root.
  • /absolute work — when the flake is a genuine product-code race needing real design.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Vulnerability and security scan (defensive, your own repo): dependency CVEs plus risky code patterns (secrets, injection, weak authz), severity x reachability triaged and remediated without suppressing. Complements the built-in /security-review. Triggers on "absolute audit", "security audit", "are we vulnerable", "scan for CVEs", "check for secrets/injection", "harden this".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

Lint and typecheck debt paydown: clear pre-existing repo-wide lint/type violations and suppressions (@ts-ignore, # type: ignore) one rule per wave, fixing causes not symptoms. Runs on green main. For diff-scoped quality use absolute-simplify. Triggers on "absolute debt", "fix our lint warnings", "clear the type errors", "get to strict mode", "burn down suppressions".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

Diátaxis-driven documentation for AI coding agents: write, improve, or audit tutorials, how-tos, reference, explanation, and developer docs (README, CONTRIBUTING, ADRs). Detects the docs stack; gates on the outline before writing prose; verifies every claim against the code before it ships. Triggers on "absolute docs", "write docs", "write a tutorial", "write a README", "document this", "improve this doc", "audit our docs".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

One-time setup for absolute: interview how you want it to behave (output style, autonomy, TDD strictness, spec dir, families) + detect the stack once, then write `.absolute.config.json` (project, committed) and `~/.absolute/config.json` (user defaults + per-project overrides). Every other absolute-* command reads it instead of re-detecting; non-blocking — commands proceed without it and soft-recommend it. Triggers on "absolute init", "set up absolute", "initialize absolute", "configure absolute", "first-time setup", "remember my conventions for this repo".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

Dead code and dependency cleanup, repo-wide: unused deps, unreferenced exports, unreachable code, orphaned files — removed only with tool evidence, in reversible waves. Runs on green main. For diff-scoped cleanup use absolute-simplify. Triggers on "absolute prune", "remove dead code", "find unused deps/exports", "what can we delete", "clean up orphaned files".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

Use when the user wants to simplify, clean up, refactor, tidy, or refine code — their staged/unstaged git changes or a target file/path. Reduces complexity, flattens nesting, removes redundancy and dead code, scores each change by value (holding low-value churn), then runs tests to prove nothing broke. Invoke on: "simplify", "simplify this", "simplify my code/changes", "clean up", "clean this up", "clean up my changes", "refactor this", "make this cleaner", "tidy this up", "reduce complexity", "flatten this", "remove dead code", "make it more readable", "polish before commit", or "absolute simplify". Acts on your working diff; for repo-wide dead code use absolute-prune; for lint/type debt use absolute-debt.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

maddhruv/absolute2212026년 7월 6일 업데이트

maddhruv의 스킬 모두 보기

이 스킬의 문제 신고