跳到正文
cccskills
免费在 GitHub 公开

numerics-bisect

Use when imp outputs differ between two paths that should agree - chunk sizes, prefill vs decode, prefix-cache resend, two engines (imp vs llama.cpp vs HF), two builds or kernels - "output changes with chunk size", "first token differs", "logprobs moved", "which layer diverges", "who is right, imp or llama.cpp", "near-tie". Do NOT use for repetition loops or garbage output (check-degeneration), a new arch that loads wrong (add-model-arch), or perf (benchmark-cuda).

中文简介正在准备中。当前显示原文说明。

查看安装方法

包含的文件(1)

  • SKILL.md3.0 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Numerics bisect - imp

Order is fixed: ids, logprobs, hidden states, reference. Each rung is cheaper than the next and kills a class of false leads.

Rungs

#StepToolDecision
1Compare token ids of both prompts firstimp-cli / server token ids vs the other pathids differ = tokenizer or template bug (add-model-arch step 5), stop here
2Logprob grid over the axis (chunk size, engine, build)first-token logprobs per arm; chunk axis via --prefill-chunk-size <n> (runtime.prefill_chunk_size); raw logits via diagnostics.dump_logits_dira spread across the axis is the signal; greedy-text equality alone amplifies near-ties
3Hidden-state dumps per layerdiagnostics.dump_hidden_dir + tools/analysis/layer_ab_diff.py (two imp runs) or tools/analysis/layer_diff.py (vs llama.cpp)first block with non-zero added divergence (rel@out - rel@in) owns the bug
4Reference arbitrationHF transformers fp32 (GGUF: from_pretrained(..., gguf_file=...), dequantized) plus llama.cpp with -fa offthe arm that matches both references is right
5Judge near-tie tokens by multi-token NLLimp-cli --perplexity tools/analysis/ppl_corpus_45k.txt (rebuild with tools/analysis/make_ppl_corpus.sh), --set runtime.deterministic=true both armsnever by one token: llama.cpp FA on vs off alone moved a near-tie token by 0.082

Dump traps

TrapFix
[DUMP_NPY] open faileddocker --user $(id -u):$(id -g) so the container can write the mounted dump dir
Dumps per stage, overwritten per forwardnames carry layer, step and n, not the prefill offset: imp_L<layer>_step<step>_n<n>.bin (src/exec/executor_forward.cu), imp_step<step>_L<layer>_<tag>.npy per stage (src/exec/executor_debug.h); prefill is step 0, so chunks of equal n overwrite each other: copy the dir between forwards and key it by prefill offset for chunk studies
Default layer setdiagnostics.dump_hidden_dir=<path> dumps layers 0/5/15/29; =all dumps every layer into /tmp

Known causes of chunk-dependent rows

  • Kernel variant chosen by row count (sm120-cuda-expert numerics rules, #2152).
  • MoE router GEMM on cuBLAS picked its algorithm by n; now gemm_gate_fp32_rows (#2167). The NVFP4 act quantize is row-local.
  • hd=512 prefill attention (Gemma-4 global layers) on cuBLAS by n; runtime.deterministic runs the fixed-order FMHA (#2167).
  • Resend check: tools/analysis/prefix_resend/prefix_resend_probe.py (check-degeneration, from #2171).

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Use when adding support for a new model architecture to imp, porting a model family, or debugging a model that loads but produces wrong output - "add support for <model>", "new arch", loader detection, chat template, tokenizer parity, RoPE variant, "outputs garbage", "prompt-blind", "digits scrambled", "NaN logits", "describes a different picture", "does it fit in VRAM". Do NOT use for kernel performance (sm120-cuda-expert) or quant-format questions (quant-formats).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

Use when benchmarking, profiling, or A/B-testing CUDA kernels or end-to-end perf in the imp inference engine on RTX 5090 (sm_120), including refreshing tests/perf_baseline.json or publishing numbers to docs/BENCHMARKS.md and the README. Triggers on "benchmark kernel", "profile cuda", "ncu", "nsys", "kernel timing", "kernel sum", "occupancy", "bandwidth bound", "compute bound", "roofline", "perf baseline", "is this regression real", "decode dropped", "aggregate throughput", "two-image A/B", "prefill kernel A/B". Do NOT use for writing/optimizing kernel code (sm120-cuda-expert) or output-quality checks (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

Use when building imp, running its test suite, checking CI status, or debugging build/test failures - "make build", "run the tests", "test-gpu", "verify-fast", GTEST_FILTER, Docker/CUDA toolchain, dependency bumps, "CI is red/blocked", "which gate failed", stale objects / segfault after a header edit, hook edits, docker-entrypoint env vars, determinism or perplexity checks. Do NOT use for benchmarking/profiling (benchmark-cuda) or output-quality batteries (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

Use when verifying that a model in the imp inference engine produces coherent output without repetition loops, token-stuck states, or state corruption across turns or streams. Triggers on "degenerates", "check degeneration", "repetition loop", "own own own", "stuck token", "empty content", "multi-turn regression", "does it still work", "NIAH", and after enabling CUDA graphs / changing forward pass / MoE routing / KV cache or KV dtype / GDN state or scan / sparse attention / PDL / speculation (MTP, n-gram) / ragged prefill / batched-decode kernels (smallm, producer quantize) / FA2 softmax.

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

Use when a question is about *structure* rather than text - who calls or launches a symbol, where it is defined, what a change would reach, whether something is dead, how a request gets from the API to a kernel. Triggers on "who calls", "who launches this kernel", "where is X defined", "what breaks if I change", "is this still used", "is this dead", "trace the path from X to Y", "blast radius", "what depends on this header". Do NOT use for free-text search (`rg` is better and cheaper) or to open a file whose path you already know.

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

Use when auditing the imp codebase for structural debt, dead code, god-objects/files, duplication, flag sprawl, or deciding whether a cleanup is worth shipping - "structure audit", "tech debt", "is this still used", "dead code", "refactor for clarity", "should we split this file", "remove this flag", "file size gate red", "alloc sites gate red". Do NOT use for build/test mechanics (building-and-testing), perf/kernel work (benchmark-cuda / sm120-cuda-expert), or output quality (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

查看 kekzl 的全部技能

报告此技能的问题