“tensor parallelism”的搜索结果

Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor, scatter_tensor, domain parallelism, sequence/spatial sharding, ring attention, DeviceMesh + DDP/FSDP2 hybrid parallelism, or physicsnemo.domain_parallel. Do NOT use for generic PyTorch DDP/FSDP setup without domain parallelism, picking a PhysicsNeMo model or example (use physicsnemo-discover), or non-distributed training questions.

中文简介正在准备中。当前显示原文说明。

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

中文简介正在准备中。当前显示英语简介。

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

中文简介正在准备中。当前显示英语简介。

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.

中文简介正在准备中。当前显示原文说明。

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

中文简介正在准备中。当前显示英语简介。

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

中文简介正在准备中。当前显示英语简介。

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.

中文简介正在准备中。当前显示原文说明。

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

中文简介正在准备中。当前显示原文说明。

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

中文简介正在准备中。当前显示英语简介。

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

中文简介正在准备中。当前显示英语简介。

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Set up large language model pretraining with TorchTitan and PyTorch. Configure training across multiple GPUs, combine four ways of splitting the workload, enable Float8 calculations on H100 hardware, and manage checkpoints for saving and resuming training.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on a single node
• Launching training across multiple nodes with SLURM
• Configuring Float8 training for H100 GPUs

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.

中文简介正在准备中。当前显示原文说明。

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

中文简介正在准备中。当前显示原文说明。

Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.

中文简介正在准备中。当前显示原文说明。

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

中文简介正在准备中。当前显示原文说明。

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

中文简介正在准备中。当前显示原文说明。