“fsdp”的搜索结果

Get guidance on distributed training with PyTorch FSDP, which splits model parameters across devices. The skill covers implementation, mixed precision, moving data to CPU memory, communication setup, and debugging distributed training code.

中文简介正在准备中。当前显示英语简介。

• Implement distributed training with FSDP
• Review mixed precision and CPU offloading settings
• Choose a distributed communication backend

Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.

中文简介正在准备中。当前显示英语简介。

• Training a model that exceeds one GPU's memory
• Adding FSDP2 to an existing training script
• Configuring mixed precision and CPU offloading

Implement and debug large-model training with PyTorch FSDP, which splits training state across devices. Consult documentation and code patterns for sharding choices, mixed precision, and checkpoints when building or reviewing distributed training code.

中文简介正在准备中。当前显示英语简介。

• Implement distributed training for large models
• Compare FSDP wrapping and sharding patterns
• Review mixed-precision and checkpoint examples

Adapt PyTorch training scripts for multiple GPUs or machines with Accelerate. This skill guides code changes, configuration, and launch commands, including mixed precision, which combines numerical formats, and DeepSpeed or FSDP integration for distributed training.

中文简介正在准备中。当前显示英语简介。

• Moving single-GPU training to multiple GPUs
• Configuring mixed precision training
• Setting up DeepSpeed or FSDP

Adapt existing PyTorch training scripts to use multiple GPUs or machines with limited code changes. The skill guides configuration, launch commands, mixed precision, which uses lower-precision numbers, and DeepSpeed or FSDP settings for distributing training work and memory.

中文简介正在准备中。当前显示英语简介。

• Move a single-GPU training script to multiple GPUs
• Configure mixed precision training
• Set up DeepSpeed or FSDP to reduce memory use

Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.

中文简介正在准备中。当前显示英语简介。

• Adapt a single-GPU training script for multiple GPUs
• Configure mixed precision to reduce training memory use
• Set up training with DeepSpeed or FSDP

Orientação especializada para treinamento com Fully Sharded Data Parallel do PyTorch (FSDP) - sharding de parâmetros, precisão mista, CPU offloading, FSDP2

中文简介正在准备中。当前显示原文说明。

Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2

中文简介正在准备中。当前显示原文说明。

Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2

中文简介正在准备中。当前显示原文说明。

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Organize PyTorch training code with LightningModule and Trainer. This skill guides training, validation, logging, model checkpoint saving, early stopping when progress stalls, and configuration for running the same model across multiple GPUs.

中文简介正在准备中。当前显示英语简介。

• Organize existing PyTorch training code
• Add validation and metric logging
• Save the best models and stop stalled training

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor, scatter_tensor, domain parallelism, sequence/spatial sharding, ring attention, DeviceMesh + DDP/FSDP2 hybrid parallelism, or physicsnemo.domain_parallel. Do NOT use for generic PyTorch DDP/FSDP setup without domain parallelism, picking a PhysicsNeMo model or example (use physicsnemo-discover), or non-distributed training questions.

中文简介正在准备中。当前显示原文说明。

Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2

中文简介正在准备中。当前显示原文说明。

Organize PyTorch models and data pipelines with PyTorch Lightning, automate training, save checkpoints, and track experiments. Includes templates and guidance for configuring training across multiple GPUs or TPUs.

中文简介正在准备中。当前显示英语简介。

• Restructure PyTorch training code into LightningModules
• Build reusable data preparation and loading pipelines
• Configure checkpoints, early stopping, and experiment logs

Organize PyTorch training code with LightningModule and Trainer. Follow examples for validation, logging, model saving, and distributed training across multiple GPUs, with callbacks that add monitoring or stop training when progress stalls.

中文简介正在准备中。当前显示英语简介。

• Converting an existing PyTorch training loop
• Adding validation, logs, and model checkpoints
• Configuring training across multiple GPUs

Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.

中文简介正在准备中。当前显示原文说明。

Guide for selecting and configuring distributed training strategies in NeMo AutoModel, including FSDP2, Megatron FSDP, DDP, and parallelism settings.

中文简介正在准备中。当前显示原文说明。

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

中文简介正在准备中。当前显示原文说明。

Fornece treinamento distribuído de LLM nativo do PyTorch usando torchtitan com paralelismo 4D (FSDP2, TP, PP, CP). Use para fazer pré-treinamento de Llama 3.1, DeepSeek V3 ou modelos customizados em escala de 8 a 512+ GPUs com Float8, torch.compile e checkpoint distribuído.

中文简介正在准备中。当前显示原文说明。

Framework PyTorch de alto nível com classe Trainer, treinamento distribuído automático (DDP/FSDP/DeepSpeed), sistema de callbacks e boilerplate mínimo. Escala de laptop para supercomputador com o mesmo código. Use quando quiser loops de treinamento limpos com boas práticas integradas.

中文简介正在准备中。当前显示原文说明。

API de treinamento distribuído mais simples. 4 linhas para adicionar suporte distribuído a qualquer script PyTorch. API unificada para DeepSpeed/FSDP/Megatron/DDP. Posicionamento automático de device, precisão mista (FP16/BF16/FP8). Config interativo, comando de launch único. Padrão do ecossistema HuggingFace.

中文简介正在准备中。当前显示原文说明。

High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.

中文简介正在准备中。当前显示原文说明。

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

中文简介正在准备中。当前显示原文说明。