Skip to content
cccskills
FreePublished on GitHub

cpu-kernel

Develop or port CPU kernels, the low-level calculation routines used by MNN. Follow staged implementation guidance, check data layout agreements, register functions, and verify correctness across processor instruction sets and numerical precision settings.

OriginalShow the original description

MNN CPU 后端 kernel 开发分支(`skills/cpu/` 下,另一分支是 `cpu/optimize` 性能归因)。

覆盖标量 oracle → C++ SIMD → intrinsic → 汇编的四级实现阶梯、pack/kernel ABI 契约(tile、cell stride、后处理参数)、CoreFunctions 派发表注册与二级表安全构造、跨 ISA × 精度的正确性门禁,以及 AArch64 / x86_64 / RISC-V 三份实现参考。

为 CPU 后端新写或移植 kernel(NEON / SSE / AVX / RVV intrinsic 或 .S 汇编)、新增一层 ISA、改 pack mode 或 tile 参数、把 kernel 挂进函数表时使用。已定位到 kernel 才进本分支。

See how to install

Good for

  • Implementing or porting an MNN CPU kernel
  • Adding support for a processor instruction set
  • Changing packed data layouts or tile sizes
  • Registering kernels and planning correctness checks

Files included (7)

  • SKILL.md9.7 KB
  • arch/arm.md45.5 KB
  • arch/riscv.md18.7 KB
  • arch/x86_64.md24.1 KB
  • correctness-gate.md11.1 KB
  • dispatch-and-register.md13.8 KB
  • pack-and-abi.md11.9 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

MNN CPU Kernel 开发 Skill

触发条件:要为 CPU 后端写一个 kernel(intrinsic 或汇编)、新增一层 ISA 支持、 改 pack mode / tile 参数、或把新函数挂进 CoreFunctions 派发表。

这个 skill 与相邻 skill 的分界

三份 skill 回答三个不同的问题,混用会导致在错的地方找答案:

skill回答的问题独占内容
add-new-op这个算子在框架里存在吗MNN.fbs schema、source/shape/Shape*.cpp、source/geometry/Geometry*.cpp 几何拆解、test/op/*Test.cpp、register.py
cpu/kernel(本分册)这条 kernel 怎么写对、怎么被选中实现阶梯、pack/ABI 契约、tile 契约、派发表注册与二级表安全写法、跨 ISA × 精度正确性门禁
cpu/optimize为什么慢、该改哪一层分层诊断、线程/调度、布局/内存、benchmark 纪律、env 开关、跨层不一致的事后定位

两条最容易走错的边界:

  • 算子还不存在 → 先去 add-new-op 把 schema / shape / geometry / op test 建起来, 再回本 skill 写 kernel。反过来先写 kernel 会发现无处注册。
  • "运行时到底选中了哪条路径、为什么退到慢路径、能力位是不是 0" → 这是诊断, 在诊断面文档 cpu/optimize/arch/arm.md §三 / cpu/optimize/arch/x86_64.md §三 / cpu/optimize/arch/riscv.md §三。 本 skill 只管**"怎样正确注册,使它能被选中"**。

没有性能数据就不要进本 skill。 手写 kernel 是最贵的一层。先用 cpu/optimize 的 投入决策门量化"改这一层的天花板在哪"——已经跑到 82% roofline 的 kernel 再写一版汇编是白干。

五条 ISA 无关铁律

三条各侧共用的纪律(先确认路径再谈性能 / 先正确再加速 / 数字必须带维度标签)在 cpu/SKILL.md「三条共用的前置纪律」,本文不复述。下面五条是 kernel 开发独有的:

  1. 「先正确再加速」在本分册的落地形式是四级阶梯,每级有明确退出条件,不许跳级。

    级做什么退出条件(满足才允许进下一级)
    A标量参考实现(oracle)在四个分层比较点上对齐——correctness-gate.md §2.1。int8 多数场景不用自己写,仓库自带标量版
    BC++ SIMD / 寄存器模拟与 A 在同样四个点上一致。分组必须与目标指令同构(sdot 的 lane 广播语义见 arch/arm.md §4.2)。简单 elementwise 可豁免此级
    Cintrinsic 实现目标 ISA 档与 oracle 对齐 + correctness-gate.md 最小矩阵通过。多数 kernel 应该停在这里,x86_64 侧更是以 intrinsic 为主力形态
    D最小汇编correctness-gate.md 完整矩阵(ISA × 精度 × 线程档)+ dispatch-and-register.md §五 注册面逐条核过

    进 D 的判据只有一条:intrinsic 已实测不够(编译器寄存器分配不理想、需精确控制 unpack 指令预算、需手写软流水),而不是"汇编应该更快"。进 D 之前先回答 arch/arm.md §3.5 的五个 live range 问题;一次只迁一个 tile 或一条 ISA 路径, 保留旧的安全路径。性能已达标就停在 C。

  2. 优先复用已有 CoreFunctions。 普通算子先拆成现有入口的组合,只有覆盖不了热点才新增 Vec/intrinsic/asm。入口清单见 cpu/optimize/SKILL.md「CoreFunctions 复用清单」 (全树唯一一份)。复用前必须逐项核对语义(参数含义、layout、转置、归一化方式、 in-place 安全性、tail 行为、量化后处理),不能只看函数名。

  3. Executor 只编排,ISA kernel 必须下沉。 CPUXxx.cpp 负责参数解析、buffer、线程划分和 fallback, 不直接写 NEON/SSE/AVX intrinsic 或汇编。SIMD 实现放 source/backend/cpu/compute/ 或对应架构目录, 经 CoreFunctions 入口分发。扩签名时同步检查所有架构实现和调用点。

  4. pack mode、tile 参数与 kernel 指针必须同时改。 packer、cell stride、MNNGetGemmUnit、 weight reorder、mixed/online reorder 选择、kernel 注册是一个原子集合。改一个不改其余, 形状仍然"合法",症状是能跑、不崩、单测过、只是模型质量变差——本仓库最难发现的一类 bug。 必须同源的五个量与要落笔的七处,见 pack-and-abi.md §一、§四。

  5. 寄存器生命周期表先于 unroll。 加 unroll、hoist 常量、复用临时寄存器之前先写 live range 表。 min/max、scale、bias、zero point、accumulator、unpack 常量都不能被 postprocess 之前的临时逻辑覆盖。

  6. 名字必须等于真实物理量。 改任何算子 / kernel 代码时,先核对沿途变量、字段、宏、asm 注释里的 名字是否就是它实际装的那个量(不只是自己新加的名字)——blockSize 是"块内元素数"还是"每行块数"、 step 是元素数还是字节数、分档判据的自变量是通道数还是 tile 数。名字骗人不会报错:编译、对拍、 单测都过,但基于它写的分档、阈值、pack 门控会一起错。名字与实际指针算术不符就改名,并全仓确认零残留。

任务 → 文档索引

我要做的事先读
从零实现一个热点 kernel铁律 1 的阶梯表定顺序,再按目标架构读 arch/ 对应那份
决定"该不该进汇编"铁律 1(进 D 的判据)
改 pack layout / tile / cell stride / weight reorderpack-and-abi.md
低 bit(w2/w3/w4)权重读取与 metadata 步进pack-and-abi.md §三
把新函数挂进函数表 / 新增一层 ISAdispatch-and-register.md
给 CoreFunctions 加字段或改签名dispatch-and-register.md §三(二级表构造)
决定要跑哪些正确性组合correctness-gate.md
写 AArch64 intrinsic / .Sarch/arm.md
写 x86_64 intrinsic / .Sarch/x86_64.md
写 RVV / 厂商矩阵扩展 kernelarch/riscv.md
kernel 写对了但慢出本分册,去 cpu/optimize
kernel 写错了、要定位cpu/optimize/bugfix.md + general-debug

ISA 选择表:先确认目标是哪一档

同一份 C++ 代码在不同档位下 tile、pack、精度语义甚至 kernel 前置条件都不同。 动手前先写清楚这次覆盖哪几格,不要用一格代表全部:

平台轴档位详见
ARM架构级别aarch32 / aarch64 低于 v8.2 / v8.2 sdot / v8.6 +smmla / v9.2 +SME2(累积)cpu/optimize/arch/arm.md §1.1、§2.2
ARM精度fp32 / fp16(arm82,第二张表)/ bf16同上 §1.2、§2.1
x86_64ISASSE / AVX2 / AVX+FMA / AVX512 No-VNNI / AVX512 VNNIcpu/optimize/arch/x86_64.md §一、§2.1
RISC-VISA标量 / 标准 RVV / 厂商矩阵扩展(IME2 等,构建期互斥)cpu/optimize/arch/riscv.md §一、§二;实现面 arch/riscv.md

ARM 侧两条轴是相乘关系,要覆盖的是格子不是档位:架构级别决定 int8 tile,精度决定 bytes / pack / 走哪张表,fp16 的 int8 档是从当前架构级别继承来的(不是第五档 ISA)。 x86_64 侧只有一条 ISA 轴,原因见下。

这些档位各由哪张表承载、三侧结构差在哪里(第二张表按什么分、二级表怎么构造、 Precision_Low 语义、ISA A/B 怎么做、MNN_CPU_USE_DEFAULT_BACKEND 行为)见 cpu/SKILL.md「三侧不同构对照表」——全树只有那一份,本文不复述,也不要把一侧的心智模型套到另一侧。

相关 skills

  • add-new-op:算子在框架里的注册面(schema / shape / geometry / op test)。
  • cpu/optimize:性能诊断、线程与调度、布局与内存、跨层不一致的事后定位。
  • cpu/shared/riscv-remote-validation.md:RISC-V 开发板交叉编译、板端正确性与性能实验的专属纪律。
  • general-debug(skill 名 bugfix):按症状分流的排查入口,下分七册(内存别名、导出/量化、fp16 值域、GPU 越界、kernel 隐式假设、陈旧缓存、逐 run 非确定)。
  • test-ci:跑回归 / CI / 真机 benchmark。

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

add-new-op

FreeSummary

Add new operators to MNN through a staged workflow covering data definitions, output shapes, computation, and tests. It prioritizes combining existing operators, with CPU implementations and additional hardware backends used when needed.

  • Adding a new operator to MNN
  • Implementing output shape calculations
  • Combining existing operators for new operations
alibaba/MNN16.2KUpdated Oct 16, 2026

bugfix

FreeSummary

Triage incorrect MNN outputs and regressions by matching symptoms to focused debugging guides. Use backend comparisons, minimal reproductions, and controlled experiments to investigate memory issues, quantization, concurrency, numeric limits, caches, and inconsistent results across runs.

  • Investigating garbled or degraded MNN output
  • Comparing results across execution backends
  • Tracking regressions after optimization changes
alibaba/MNN16.2KUpdated Oct 16, 2026

cpu

FreeSummary

Find the right workflow for MNN CPU work, from investigating slow execution to implementing calculation routines and planning tests. Compare ARM, x86_64, and RISC-V backend structures before choosing the relevant guidance.

  • Choosing where to start an MNN CPU performance investigation
  • Finding guidance for CPU kernel implementation
  • Comparing ARM, x86_64, and RISC-V backend structures
alibaba/MNN16.2KUpdated Oct 16, 2026

cpu-optimize

FreeSummary

Find why MNN CPU inference is slow by examining threads, scheduling, memory layout, and the processor instruction path actually used. Operator-level measurements and comparisons with other inference frameworks help identify which layer needs improvement across ARM, x86_64, and RISC-V.

  • Investigate slow CPU operators or LLM inference
  • Diagnose thread count and memory usage anomalies
  • Locate the cause of a performance regression
alibaba/MNN16.2KUpdated Oct 16, 2026

Optimize and troubleshoot MNN's Hexagon DSP backend, which runs computation on a dedicated processor. The workflow uses device measurements to identify bottlenecks, compare v79 and v81 architectures, verify correctness, and investigate crashes or memory usage.

  • Finding bottlenecks in Hexagon operators
  • Checking changes on v79 and v81 devices
  • Investigating cDSP crashes and device disconnects
alibaba/MNN16.2KUpdated Oct 16, 2026

metal-optimize

FreeSummary

Guide MNN Metal backend development and optimization, from finding performance bottlenecks to changing GPU computation, combining operators, and adjusting execution scheduling. A task-based document index directs work toward relevant implementation rules, benchmarks, and correctness checks.

  • Finding bottlenecks in Metal inference
  • Adding or modifying Metal kernels
  • Combining operators into one GPU dispatch
alibaba/MNN16.2KUpdated Oct 16, 2026

See all skills by alibaba

Report a problem with this skill