22 skills · sorted by relevance
Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.
Get guidance on distributed training with PyTorch FSDP, which splits model parameters across devices. The skill covers implementation, mixed precision, moving data to CPU memory, communication setup, and debugging distributed training code.
Adapt PyTorch training scripts for multiple GPUs or machines with Accelerate. This skill guides code changes, configuration, and launch commands, including mixed precision, which combines numerical formats, and DeepSpeed or FSDP integration for distributed training.
Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.
Adapt existing PyTorch training scripts to use multiple GPUs or machines with limited code changes. The skill guides configuration, launch commands, mixed precision, which uses lower-precision numbers, and DeepSpeed or FSDP settings for distributing training work and memory.
Implement and debug large-model training with PyTorch FSDP, which splits training state across devices. Consult documentation and code patterns for sharding choices, mixed precision, and checkpoints when building or reviewing distributed training code.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.
Audits arithmetic code for mixed units, decimal precision mistakes, and scaling errors. Adds comments identifying each value’s units and produces a summary of findings, unit definitions, and files reviewed.
Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.
Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.
Train GPT-2 scale models (~124M parameters) efficiently on a single GPU. Covers GPT-124M architecture, tokenized dataset loading (e.g., HuggingFace Hub shards), modern optimizers (Muon, AdamW), mixed precision training, and training loop implementation.
The English summary is being prepared. Showing the original description.
Battle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discovery, spatial omics, genomics. Covers training loops, optimizer selection (AdamW, Muon), LR scheduling, mixed precision, debugging, and systematic experimentation. Use when training or fine-tuning neural networks, debugging loss spikes or OOM, choosing architectures, or optimizing GPU throughput.
The English summary is being prepared. Showing the original description.
Use when writing or changing PyTorch code — model, Dataset/DataLoader, training and evaluation loops, determinism, mixed precision, torch.compile, checkpoints with safetensors, Hugging Face models, ONNX export, and tests that run on CPU
The English summary is being prepared. Showing the original description.
API de treinamento distribuído mais simples. 4 linhas para adicionar suporte distribuído a qualquer script PyTorch. API unificada para DeepSpeed/FSDP/Megatron/DDP. Posicionamento automático de device, precisão mista (FP16/BF16/FP8). Config interativo, comando de launch único. Padrão do ecossistema HuggingFace.
The English summary is being prepared. Showing the original description.
Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention
The English summary is being prepared. Showing the original description.
Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.