Results for “tensor parallel”

Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor, scatter_tensor, domain parallelism, sequence/spatial sharding, ring attention, DeviceMesh + DDP/FSDP2 hybrid parallelism, or physicsnemo.domain_parallel. Do NOT use for generic PyTorch DDP/FSDP setup without domain parallelism, picking a PhysicsNeMo model or example (use physicsnemo-discover), or non-distributed training questions.

The English summary is being prepared. Showing the original description.

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.

The English summary is being prepared. Showing the original description.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

The English summary is being prepared. Showing the original description.

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.

• Training a model that exceeds one GPU's memory
• Adding FSDP2 to an existing training script
• Configuring mixed precision and CPU offloading

Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.

The English summary is being prepared. Showing the original description.

Fornece treinamento distribuído de LLM nativo do PyTorch usando torchtitan com paralelismo 4D (FSDP2, TP, PP, CP). Use para fazer pré-treinamento de Llama 3.1, DeepSeek V3 ou modelos customizados em escala de 8 a 512+ GPUs com Float8, torch.compile e checkpoint distribuído.

The English summary is being prepared. Showing the original description.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

The English summary is being prepared. Showing the original description.

Set up large language model pretraining with TorchTitan and PyTorch. Configure training across multiple GPUs, combine four ways of splitting the workload, enable Float8 calculations on H100 hardware, and manage checkpoints for saving and resuming training.

• Pretraining Llama 3.1 8B on a single node
• Launching training across multiple nodes with SLURM
• Configuring Float8 training for H100 GPUs

Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.

The English summary is being prepared. Showing the original description.

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

The English summary is being prepared. Showing the original description.

Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.

The English summary is being prepared. Showing the original description.

VAE/TAE 解码段优化（**计算 + 通信同技能**）：判断解码器能否沿某个轴切到多张卡、怎么切才是逐位精确、切了值不值；给出可复用的等价性证明流程、状态携带切分代码模式与交换预算判据。范围**含 VAE encode（待补：当前无内容，暂不纳入正文）**。当用户问“VAE/TAE 解码能不能分卡/切分/分摊”“8 张卡重复解码能不能省”“按帧轴还是空间轴切”“切完结果会不会变”“怎么证明等价”“all-gather 拼回来对不对”，或任何“多卡把解码摊开、又要保证输出不变”的场合都应触发；即使用户没说“分片”二字，只要出现“解码每卡都解一遍”“解码太慢想摊到多卡”“那部分重复计算能不能省”也应触发。覆盖：跨轴耦合算子审计（含递归 MemBlock 前缀依赖的否证）、潜帧边界切分＋每块末帧状态携带、与 rank 无关的判定（否则死锁）、交换预算核算、已知静默陷阱（all_gather_into_tensor 填栈副本、repeat_interleave 多分配整张量、rank 选举式参与）、有损分片的精确性前置条件与自动修复、CPU 真值→分段 bitwise→端到端 md5→帧级健康指标的四道验证闸、分片“到底有没有生效”的状态量判定，以及“切分顺带绕过设备缺陷”的用法。边界（本技能只引用不复制）：计算手段与能力供给见 `pattern-dev` / `operator-dev`，DiT 侧选档参照 `dit-perf-opt`；通信方法论（形态/掩盖/通信账/归因）见 `dit-parallel-opt`；一致性判据见 `accuracy-gate`；数字入库见 `perf-gate`；交付搬运与装载预热等 host 固定开销见 `host-opt`（不属本技能）。

The English summary is being prepared. Showing the original description.

Distributed machine learning, data mining, and iterative HPC with Exasol. Covers end-to-end ML pipelines (DISTRIBUTE BY + SET scripts + BucketFS), per-entity federated training with partial_fit and ctx.reset(), batch inference, map-reduce ensemble training, distributed ensemble and SON algorithm for frequent itemset mining (Apriori, FP-Growth, association rules, market-basket analysis), Lua execute script orchestration for iterative algorithms (k-means, SGD, gradient descent), scikit-learn model training, parallel hyperparameter search, per-entity forecasting, anomaly detection, model lifecycle in BucketFS (pickle/joblib/ONNX versioning), GPU acceleration via CUDA SLCs (PyTorch/TensorFlow/RAPIDS), and ML-specific performance tuning (skew, OOM, multi-pass chunking).

The English summary is being prepared. Showing the original description.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

The English summary is being prepared. Showing the original description.