40 skills · sorted by relevance
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
The English summary is being prepared. Showing the original description.
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.
Guides LLM fine-tuning with Axolotl, from YAML configuration and dataset formats to distributed training across GPUs. Covers LoRA and QLoRA, methods that train on preferences, multimodal support, and troubleshooting using documentation references and code examples.
Deep learning framework (PyTorch Lightning). Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard), distributed training (DDP, FSDP, DeepSpeed), for scalable neural network training.
The English summary is being prepared. Showing the original description.
Practical guidance for training MoE VLMs in Megatron Bridge. Compares FSDP and 3D-parallel approaches, using rounded lessons from Qwen3-VL, Qwen3-Next, and other multimodal experiments.
The English summary is being prepared. Showing the original description.
Set up infrastructure for fine-tuning LLMs with QLoRA, LoRA, and full fine-tuning using Hugging Face TRL, Axolotl, and distributed training with DeepSpeed or FSDP. Covers dataset prep, training runs, and model export.
The English summary is being prepared. Showing the original description.
Use Hugging Face Accelerate for PyTorch training-loop migration, distributed launch/configuration, DeepSpeed/FSDP/TPU backend setup, big-model inference/offload, checkpointing, tracking, and troubleshooting.
The English summary is being prepared. Showing the original description.
Think and work like an expert Deep Learning Scientist. Use when a task calls for Deep Learning Scientist judgment. Reasons from CNN/Transformer inductive bias, Li et al. loss landscapes, grokking/mode connectivity, and Kaplan/Chinchilla scaling (~20 tokens/param); designs ResNet/ViT/DiT/MoE/FlashAttention stacks with FLOPs-matched ablations; trains AdamW+cosine/WSD via Megatron-FSDP/DeepSpeed; evaluates FID/MMLU-Pro/MMLU-CF with lm-eval decontamination and Pineau/NeurIPS reproducibility checklists.
The English summary is being prepared. Showing the original description.
Use when training models across multiple GPUs or nodes, handling large models that don't fit in memory, or optimizing training throughput - covers DDP, FSDP, DeepSpeed ZeRO, model/data parallelism, and gradient checkpointingUse when ", " mentioned.
The English summary is being prepared. Showing the original description.
Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.
The English summary is being prepared. Showing the original description.
对 msMemScope 显存数据做全链路处理：根据用户的显存问题诉求（OOM、显存泄漏、显存占比拆解、Step 间对比等），指导并帮助用户以最合适的 msMemScope 采集配置完成显存数据采集（支持命令行、Python 接口、mstx 打点三种采集方式，覆盖训练、推理 vLLM-Ascend、FSDP、强化学习 verl 等场景）；对已采集的 memscope_dump_*.csv/db 数据做系统性解读与问题诊断（整体显存曲线趋势分析、显存使用大头按模块/组件拆解分析、显存峰值点各模块与流程占比分析，以及对显存泄漏、OOM 等典型问题的根因定位与原因分析）。
The English summary is being prepared. Showing the original description.
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.
The English summary is being prepared. Showing the original description.
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.