跳到正文
cccskills
免费在 GitHub 公开

parallelism-planning

Plan how to split AI models and workloads across multiple GPUs. Compare memory needs, computation, communication, and waiting time to estimate completion times and choose a layout suited to response time, throughput, or cost goals.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

比较数据、张量、序列、上下文、流水和专家并行的容量与通信代价。用于决定多卡切分和超节点规模。

查看安装方法

适用场景

  • Running models that exceed a single GPU's memory
  • Comparing ways to split work across GPUs
  • Estimating communication costs within and between servers
  • Balancing response time, throughput, and request cost

包含的文件(1)

  • SKILL.md1.1 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

多卡并行方案

适用:单卡容量或吞吐不足,需要决定如何切分模型、状态与工作。依据 manuscripts/06-超节点.md 第 6.1—6.7 节。

  1. 明确任务是最小化单会话延迟、完成一批请求的时间,还是每请求成本;写出设备数、每卡容量和实际拓扑。
  2. 对每种候选切分,标出权重、激活、KV 的持有者与生命周期。先检验单卡容量、合法张量形状和流水所需的批量。
  3. 列出每层或每步的 collectives、字节量、轮次与经过的链路;区分超节点内互联和跨服务器路径。计算不能无限平分,串行阶段、同步与最慢卡仍在关键路径上。
  4. 比较端到端延迟、吞吐和故障域。较大实例可能缩短单请求,却减少可同时服务的独立请求数。

输出每种方案的容量、计算、通信、空泡和完成时间估算。对于跨超节点的网络路径继续使用 cluster-network-analysis。

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Compare computing devices for a defined AI model workload using memory capacity, computing speed, data transfer, energy use, and cost. Build a comparison that separates predicted performance from measurements and identifies the constraints that determine the choice.

中文简介正在准备中。当前显示英语简介。

  • Choosing hardware for a specific AI model
  • Checking whether a model fits in device memory
  • Explaining gaps between advertised and measured performance
bojieli/ai-infra-book68702026年10月1日 更新

Plan when an agent’s tool environments are created, kept available, released, and restored. Estimate peak concurrency, resource use during work and waiting, total task duration, and cost across model calls, tool execution, validation, and retries.

中文简介正在准备中。当前显示英语简介。

  • Estimating peak concurrent agent environments
  • Comparing environment retention and recreation
  • Assessing cold starts and shared environment pools
bojieli/ai-infra-book68702026年10月1日 更新

Estimate cross-server traffic, network bottlenecks, and synchronization waits for AI training or inference. Map communication onto physical network paths and assess how topology or communication changes affect the time needed to complete the job.

中文简介正在准备中。当前显示英语简介。

  • Find communication bottlenecks in distributed AI workloads
  • Compare network topologies and communication paths
  • Estimate synchronization waits under concurrent traffic
bojieli/ai-infra-book68702026年10月1日 更新

Compare ways to distribute AI inference across multiple instances, including replicas and separate input-processing and generation stages. Estimate capacity, throughput, delays, state-transfer costs, and recovery overhead to support routing and scaling decisions.

中文简介正在准备中。当前显示英语简介。

  • Compare replicas with separate inference stages
  • Plan routing around compute capacity and cached state
  • Evaluate expert placement and load imbalance
bojieli/ai-infra-book68702026年10月1日 更新

Compare running models or tasks on a device, nearby edge infrastructure, or in the cloud. Break down interaction delays, data transfers, energy use, costs, and failure recovery to assess deployment choices for multimodal and real-time agents.

中文简介正在准备中。当前显示英语简介。

  • Comparing device, edge, and cloud execution
  • Estimating interaction delays on unstable networks
  • Assessing battery use and sustainable processing speed
bojieli/ai-infra-book68702026年10月1日 更新

Compare batching, request scheduling, and memory management for a single model serving instance. Assess concurrent request capacity, response times, generation speed, and GPU time per result that meets the required quality and deadline.

中文简介正在准备中。当前显示英语简介。

  • Comparing batching and request scheduling options
  • Checking memory capacity for concurrent requests
  • Assessing cache reuse and offloading tradeoffs
bojieli/ai-infra-book68702026年10月1日 更新

查看 bojieli 的全部技能

报告此技能的问题