为给定模型负载比较加速器的容量、有效计算、内存带宽、功耗与成本。用于硬件选型或解释标称峰值与实测差异。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026
FreePublished on GitHub
model-footprint
比较模型配置或架构改变对权重容量、KV 状态、前向计算和访存的影响。用于选模型、上下文或注意力与 MoE 方案。
The English summary is being prepared. Showing the original description.
See how to installFiles included (1)
- SKILL.md1.3 KB
SKILL.md (original)
Review the instructions the agent will receive before you install.
模型架构资源账
适用：讨论长上下文、GQA/MLA、MoE、精度或 batch 变化时，先量化模型本身的资源需求。依据 manuscripts/02-模型架构.md 第 2.1—2.6 节。
- 从模型配置取得层数、隐藏维、注意力头和 KV 头、专家总数及每 token 激活数、参数精度。没有配置时标记未知，勿从参数总数臆造结构。
- 分开计算权重容量、每 token 上下文状态大小、prefill 计算、单步 decode 计算，以及完整输出期间的状态读取。区分上下文长度、输出长度和 batch 对各项的作用。
- 对注意力变体，核对实际保存和读取的状态表示；对 MoE，区分总参数的驻留容量与每 token 的激活计算、批内被访问的专家权重。
- 比较两种架构时固定任务质量、输入输出长度、并发与硬件；列出变化的量及未变化的量，再判断容量或带宽瓶颈是否迁移。
输出按 prefill、单步 decode、完整请求分列的容量、FLOPs、读写字节表。模型配置与示例计算见 calculations/README.md。系统级延迟、排队和调度交给 workload-modeling 或 inference-serving。
Reviews
No reviews yet. Tell others what you thought.
Comments
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
More skills from this repository
规划 Agent 模型调用与工具环境的创建、驻留、释放和恢复。用于并发容量、沙箱成本与完整任务时延分析。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026
估算多节点训练或推理的跨服务器流量、网络瓶颈与同步等待。用于比较拓扑、通信路径和网络优化收益。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026
比较推理副本、prefill/decode 分离、专家放置与共享 KV 的跨实例方案。用于容量规划、路由、扩缩容和故障恢复。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026
比较模型或任务放在端侧、边缘或云端时的交互延迟、网络传输、能耗与恢复。用于多模态和实时 Agent 部署。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026
为单实例推理服务比较批处理、请求调度、KV 缓存、卸载和推测解码。用于容量、吞吐、延迟和每合格请求成本的取舍。
The English summary is being prepared. Showing the original description.
bojieli/ai-infra-book6.7KUpdated Oct 1, 2026