Skip to content
cccskills
FreePublished on GitHub

msmodelslim-layer-wise-quantization

为 msModelSlim 适配器实现逐层量化(按层加载/懒加载)能力。仅在用户明确要求逐层量化或基础适配因 CPU 内存不足无法全量加载权重时使用。该特性为高阶可选项,不是基础适配必需项。

The English summary is being prepared. Showing the original description.

See how to install

Files included (4)

  • SKILL.md2.4 KB
  • references/implementation_guide.md1.1 KB
  • references/layerwise_adapter_example.py2.8 KB
  • references/workflow.md1021 B

SKILL.md (original)

Review the instructions the agent will receive before you install.

msModelSlim 逐层量化 Skill

用于在基础适配完成后,为超大模型增加“逐层量化(按层加载)”能力,以降低 CPU 内存峰值。

触发条件

  • 用户明确要求实现逐层量化/逐层加载/懒加载/按层加载。
  • 或基础适配已完成,但 CPU 内存无法全量加载权重。

必须前置条件

  • 已完成基础适配器开发(5 个基础接口可用)。
  • 已完成四步验证流程(生成测试模型 -> 全回退量化 -> 权重一致性 -> 实际量化验证)。
  • 模型目录存在 model.safetensors.index.json,且可按层定位权重。

约束声明(必须告知用户)

  • 逐层量化是高阶可选特性,不是基础适配必需项。
  • 该方案用于“内存受限时继续开发”,不保证一定适配成功。
  • 常见失败点:层构造参数不一致、权重键名不匹配、自定义模块副作用、MTP/MoE 特殊路径未覆盖。

最小实现步骤

  1. 增加权重索引与按需读取能力:
    • get_weight_map(缓存 model.safetensors.index.json)
    • get_state_dict(module, prefix)(只读取当前层所需权重)
  2. 实现按层实例化与加载:
    • load_decoder_if_not_exist(model, name, idx)
    • 缺层时按模板层构造并加载该层 state_dict。
  3. 实现逐层遍历器:
    • generate_decoder_layer(model) 按 num_hidden_layers 逐层 yield。
  4. 在 generate_model_visit 与 generate_model_forward 统一使用逐层遍历器,保持严格同序。

最小验证

  • generate_decoder_layer 能完整遍历全部层。
  • generate_model_visit 与 generate_model_forward 无层序错位。
  • 抽样 1-2 层前向结果 shape 连续、无异常。

参考资料

结束输出要求

  • 明确告知用户是否满足触发条件。
  • 若满足,给出接入点、最小改造范围与验证结果。
  • 若不满足,建议先完成基础适配与四步验证,不提前进入逐层量化。

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

当用户直接要求进行数据集压缩测评,或在量化调优过程中需要使用 AISBench 评测工具输出数据集精度等场景时使用。 本 Skill 负责:使用 AISBench 数据集压缩代码、用 RBF Kernel Herding 算法生成压缩后的数据子集(coreset),并按用户目标交付两种产物之一: ① 接入量化调优主流程:产出独立的 coreset 数据集配置(不覆盖原始数据),回传「全集 config_name + coreset config_name」给编排层,供 evaluation.yaml 切换; ② 仅数据集压缩测评(不做量化调优):基于 example 模板生成「全集测试脚本」和「子集测试脚本」两份 shell 脚本。 暂时支持 AIME 2025 和 GPQA,其他数据集需要再结合 aisbench tools目录下 herding_coreset_selector 源码进行分析。

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

Ascend cluster comparison tool. Invoke when user asks to compare two cluster datasets (DB or TEXT), or to generate cluster_analysis_output from raw profiling data via msprof-analyze and then compare.

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

专门用于 Ascend 集群 Profiling 性能数据的“快慢卡”诊断专家技能。当用户提供【集群性能数据目录/路径】并要求分析【快慢卡】、【慢节点】、【负载不均衡】或【集群瓶颈】时,必须触发此技能。该技能会自动接收集群路径,调度相关工具输出快慢卡的宏观定性与微观根因(如 Host 下发瓶颈、算子计算劣化)。

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

Analyze Ascend NPU collective communication profiling data with a DB-first workflow. Use when the user provides `cluster_analysis_output/cluster_analysis.db`, rank-level `analysis.db`, rank-level `ascend_pytorch_profiler_{rank_id}.db`, together with `profiler_info.json`, and asks about HCCL or hcom communication cost, collective communication TOP ops, wait time, slow rank/straggler, Notify Wait, bandwidth, retry, relay, SDMA/RDMA/HCCS links, communication matrix, or Ascend communication fault patterns.

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

Analyze Ascend NPU computation-side profiling data for single-card runs or a selected rank from multi-card runs. Use this skill when the user asks to diagnose computation bottlenecks, AI Core / AI Vector / AICPU hotspots, dynamic shape overhead, block dim issues, redundant TransData/Transpose/Cast, cross-stream waits, frequency/runtime-state issues, or fusion opportunities.

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

针对昇腾 NPU 上的 PyTorch 模型,识别可做融合算子替换的代码段、替换为 torch_npu 融合算子并做性能/精度验证。当用户提到融合算子、NPU 算子替换、RMSNorm/SwiGLU/RoPE/Attention 融合、torch_npu 亲和算子等关键词是,使用本 Skill。

The English summary is being prepared. Showing the original description.

kali20gakki/msAgent32Updated Oct 19, 2026

See all skills by kali20gakki

Report a problem with this skill