跳到正文
cccskills
免费在 GitHub 公开

add-model-arch

Use when adding support for a new model architecture to imp, porting a model family, or debugging a model that loads but produces wrong output - "add support for <model>", "new arch", loader detection, chat template, tokenizer parity, RoPE variant, "outputs garbage", "prompt-blind", "digits scrambled", "NaN logits", "describes a different picture", "does it fit in VRAM". Do NOT use for kernel performance (sm120-cuda-expert) or quant-format questions (quant-formats).

中文简介正在准备中。当前显示原文说明。

kekzl/imp
查看安装方法

包含的文件（1）

  • SKILL.md7.8 KB

SKILL.md（原文）

安装前可以查看交给 AI 代理的指令内容。

Adding a Model Architecture - imp

First: is it a new arch at all?

Diff config.json against a supported sibling before estimating. Qwen3.8 shipped with zero enum members (loads as QWEN35); the work was tokenizer parity, template goldens, KV dtype default and MTP head (#1750). HF reference values come from curl, never from memory.

Start with scripts/check_model.sh <hf-repo> [--nvfp4] [--bos]: download, HF BF16 PPL (CPU), imp PPL, /tokenize ids vs HF, reasoning split, json_object; one table, exit 1 on a FAIL row. A checkpoint whose rows are all ok needs only its MODELS.md row (MiniCPM5-2B, LFM2.5-24B-A2B).

Integration checklist (gpt-oss #572 is the reference PR)

StepWhereNotes
1. Enum + registryModelArch in src/model/model_arch.h; parse_model_arch (GGUF general.architecture in gguf_loader.cpp, HF architectures/model_type in hf_config_loader.cpp; arch-only config.json keys: one file in src/model/hf_config/ + a kHfConfigHooks row in arch_registry.cpp), model_arch_name, apply_arch_defaults, sampling defaults in src/model/model.cppthen ModelProfile (src/model/model_profile.h/.cpp, SSoT since #622/#623; AttnVariant { STANDARD, GEMMA4_SWA, GPTOSS_SWA, NOPE, MLA }). No new cfg.arch == X in hot paths. Add the arch to the KV-dtype lists kv_nvfp4_default_safe, kv_fp8_hint_default_safe, kv_fp8_no_hint_default_safe (evidence per family in model.cpp): a missing entry silently gets FP16 KV, which on a GDN hybrid gates context
2. Loadersrc/model/tensor_kind_matcher.cpp, weight_map.cpp; SafeTensors safetensors_loader.cpp; NVFP4 via llm_compressor_loader.cpp (compressed-tensors) or Modelopt (hf_quant_config.json)the two NVFP4 layouts have RECIPROCAL tensor scales (quant-formats)
3. Arch configmodel_config.h + apply_arch_defaultsRoPE pair layout (NeoX vs GPT-J), YaRN/rope_freq_scale, SWA layer pattern, NoPE, sinks, softcap, norm placement, MoE router
4. Chat templatefamily in src/model/chat_template_families.cpp (ChatTemplateFamily in chat_template.h), rendering in chat_template.cpp (+ jinja.cpp)a new family needs a golden pin (make chat-goldens, tests/refs/chat_template_goldens.h, nine families since #1721, three more Jinja gaps fixed in #1701; Jinja fails SILENTLY). reasoning_effort must reach ChatTemplate::apply*/render_jinja + the server snapshot: identical prompt-token counts across efforts = not threaded (#1750: 67/67 before, 41/11/53 after)
5. Tokenizer paritytemplate tests/test_tokenizer_qwen38.cpp (32/32 encode+decode vs HF), tests/test_qwen38_chat_template.cpp; harness tools/tokenizer_parity/run.sh (from #2159, branch fix/tokenizer-hf-parity until merged)BERT-family GGUFs use SPM, not WordPiece. An HF reference built from a GGUF tokenizer can split special tokens (<think> -> 6 text ids instead of 151667): compare token ids before logits
6. Kernelsonly for genuinely new ops; check src/exec/ + src/compute/ firstnew RoPE variants go into src/compute/rope_yarn.cuh (shared with MTP heads, #913)
7. Verifyloads -> greedy coherent (check-degeneration) -> imp-cli --perplexity vs HF (within ~10-20%, often closer: gpt-oss 4.68 vs bf16 4.607, #663) -> decode/prefill sanity (benchmark-cuda)PPL with runtime.deterministic=true, speculative.mtp_k=0, ppl_corpus_45k.txt (tools/analysis/make_ppl_corpus.sh)
8. Docsdocs/MODELS.md row (+ docs/BENCHMARKS.md if hero); known gaps to docs/LIMITATIONS.md; perf baseline entry if gated

A new checkpoint is UNTRUSTED INPUT: SafeTensors/tokenizer.json parsers are hardened (#1660, #1694); fuzz targets under fuzz/; no parsing shortcuts around the bounds checks.

Wrong-output fingerprints

SymptomRoot-cause classCase
Fluent but ignores the prompt ("prompt-blind")RoPE pair layout: HF SafeTensors need rope_neox=true, GGUF pre-permutes Q/KSafeTensors Llama/Mistral, #503
Words fine, digits scrambledposition encoding (NoPE layer as RoPE or vice versa)Nemotron-H rope_attn_disabled, #518
Argmax always token 0NaN logits upstream (residual overflow, bad scale)gpt-oss FP16 residual
Coherent to ~1k ctx, then garbageYaRN/rope_freq_scale inverted or fused-rope path without YaRNgpt-oss 1024x error, #572
Wrong only with chunked prefill at long ctxcontinuation-chunk path#553
Wrong language / valid-but-wrong tokensweight upload / dequant layout (MoE: weight_upload.cpp expert promotion first)Qwen3.6-35B NVFP4, #925
Garbage from token 0 (!!!)silent VRAM-alloc failure in a decode fallbackMXFP4 GDN hybrids, #935
Multimodal: describes a DIFFERENT pictureM-RoPE per-token (t,h,w) layout, src/model/mrope_positions.cppQwen3-VL
Vision fluent but generictower loaded partly or embeddings never reach the sequence: tools/analysis/vision_sight_check.py
Drift only at very long positionsYaRN float trap: __sinf/__cosf on an unreduced argument; long-ctx tests run ext_factor=0 and cannot see it#1704
Coherent-ish, drifts vs HFMLA/YaRN rope_mscale on the wrong base (1.261x); pin the transformers oracle (4.44.2 for MLA)#880
Draft accept collapses, output correctMTP head RoPE differs from the target (NeoX vs YaRN)#913
CLI fine, server brokennot arch: server-api
Which block divergesdiagnostics.dump_hidden_dir + tools/analysis/layer_diff.py (vs llama.cpp llama-eval-callback) or layer_ab_diff.py (two imp runs); dump_gdn_state_dir, dump_logits_dir0 non-finite GDN states over a 46579-token prefill on Qwen3.8

Known traps

  • rope_neox: GGUF converters pre-permute Q/K; HF SafeTensors do not.
  • swa_layers was Gemma-only hardcoded once; verify per-layer attention type on any interleaved-SWA arch.
  • The fused rope+KV-write kernel must apply the same YaRN scaling as the standalone path.
  • Arch control tokens (Harmony channels) must not land on the banned list; the spec-verify argmax applies the mask since #1796.
  • Gemma-4: per-layer rope_freqs for non-SWA layers, n_rot=hd.
  • GDN/hybrid state: BF16 storage + FP32 arithmetic is the default (gdn.state_bf16, #1776/#1778); FP16 state NaNs at depth; the old "must be FP32" was a layout bug.
  • Multimodal checkpoints wrap the LM under model.language_model.* (Qwen3.5-VL, #647): strip in the loader or every tensor "is missing".
  • Encoder/embedding archs are supported (nomic-bert #867, cosine 0.999 vs HF).
  • VRAM arithmetic: card 32 607 MiB; CUDA primary context ~1680 MiB; library reserve is a per-model MEASURED value cached in src/memory/library_reserve_cache.h (#1119; vram.library_reserve_cache, override vram.library_reserve_mb); the kMeasuredLibraryReserveBytes ~3900 MiB constant in src/memory/plan.h is the first-run fallback (measured 0 MiB on Qwen3-4B-IQ4_NL, 7460 on Qwen3-8B-Q8_0). An MTP head adds ~0.79 GiB when speculative.mtp_k auto takes it. Peaks per config: docs/internals/MEMORY.md.
  • Getting a checkpoint onto the NVFP4 path: scripts/stage-model.sh (download + imp-quantize in one command; imp fetches nothing itself).

After it works

make verify-fast; a DegenerationTest-compatible probe prompt; think-channel archs through tools/analysis/degen_suite.py (think-leak); vision through vision_sight_check.py and make test-vision.

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护，适用 Google 的隐私权政策和服务条款。

同一仓库的技能

benchmark-cuda

免费

Use when benchmarking, profiling, or A/B-testing CUDA kernels or end-to-end perf in the imp inference engine on RTX 5090 (sm_120), including refreshing tests/perf_baseline.json or publishing numbers to docs/BENCHMARKS.md and the README. Triggers on "benchmark kernel", "profile cuda", "ncu", "nsys", "kernel timing", "kernel sum", "occupancy", "bandwidth bound", "compute bound", "roofline", "perf baseline", "is this regression real", "decode dropped", "aggregate throughput", "two-image A/B", "prefill kernel A/B". Do NOT use for writing/optimizing kernel code (sm120-cuda-expert) or output-quality checks (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

building-and-testing

免费

Use when building imp, running its test suite, checking CI status, or debugging build/test failures - "make build", "run the tests", "test-gpu", "verify-fast", GTEST_FILTER, Docker/CUDA toolchain, dependency bumps, "CI is red/blocked", "which gate failed", stale objects / segfault after a header edit, hook edits, docker-entrypoint env vars, determinism or perplexity checks. Do NOT use for benchmarking/profiling (benchmark-cuda) or output-quality batteries (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

check-degeneration

免费

Use when verifying that a model in the imp inference engine produces coherent output without repetition loops, token-stuck states, or state corruption across turns or streams. Triggers on "degenerates", "check degeneration", "repetition loop", "own own own", "stuck token", "empty content", "multi-turn regression", "does it still work", "NIAH", and after enabling CUDA graphs / changing forward pass / MoE routing / KV cache or KV dtype / GDN state or scan / sparse attention / PDL / speculation (MTP, n-gram) / ragged prefill / batched-decode kernels (smallm, producer quantize) / FA2 softmax.

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

code-graph

免费

Use when a question is about *structure* rather than text - who calls or launches a symbol, where it is defined, what a change would reach, whether something is dead, how a request gets from the API to a kernel. Triggers on "who calls", "who launches this kernel", "where is X defined", "what breaks if I change", "is this still used", "is this dead", "trace the path from X to Y", "blast radius", "what depends on this header". Do NOT use for free-text search (`rg` is better and cheaper) or to open a file whose path you already know.

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

codebase-audit

免费

Use when auditing the imp codebase for structural debt, dead code, god-objects/files, duplication, flag sprawl, or deciding whether a cleanup is worth shipping - "structure audit", "tech debt", "is this still used", "dead code", "refactor for clarity", "should we split this file", "remove this flag", "file size gate red", "alloc sites gate red". Do NOT use for build/test mechanics (building-and-testing), perf/kernel work (benchmark-cuda / sm120-cuda-expert), or output quality (check-degeneration).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

docs-layers

免费

Use when writing, moving or auditing any .md in imp - deciding which file a paragraph belongs in, adding a doc, fixing a stale claim, or when docs_lint.py fails in CI. Covers the four reader layers (L0 README / L1 operators / L2 kernel devs / L3 agents), the HTML-comment metadata header, [PROV:] provenance, the single-source-of-truth map, generated perf blocks, which numbers may appear in prose, plan-doc closure. Triggers on "which doc does this go in", "docs lint failed", "add a doc", "this claim is stale", "update the README", "PROV block", "layer", "STALE.md". Do NOT use for the CHANGELOG or a release body (shipping-prs), or for code-comment accuracy (codebase-audit).

中文简介正在准备中。当前显示原文说明。

kekzl/imp452026年10月19日 更新

查看 kekzl 的全部技能

报告此技能的问题