19 skills · sorted by relevance
Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.
Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.
Get guidance on distributed training with PyTorch FSDP, which splits model parameters across devices. The skill covers implementation, mixed precision, moving data to CPU memory, communication setup, and debugging distributed training code.
Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.
Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.
Guides setup and operation of Lambda Labs GPU instances for machine learning training and inference. Covers SSH access, persistent storage, API automation, and multi-GPU or multi-node training, including how to retain checkpoints and terminate idle instances.
Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.
Guides setup and operation of Lambda Labs GPU instances for machine learning training and inference. Covers SSH access, persistent storage, API automation, and multi-GPU or multi-node training, including how to retain checkpoints and terminate idle instances.
Train Mixture of Experts models, which activate selected specialist networks for each input. The skill guides architecture, input routing, workload balancing, and GPU distribution using DeepSpeed or HuggingFace, with examples for training configuration and inference optimization.
Build and train Mixture of Experts models, which activate selected specialist networks for each input. Use DeepSpeed or HuggingFace workflows to configure routing, balance workloads, distribute experts across GPUs, and optimize inference.
Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.
The English summary is being prepared. Showing the original description.
Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention
The English summary is being prepared. Showing the original description.
Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2
The English summary is being prepared. Showing the original description.
Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.
The English summary is being prepared. Showing the original description.
Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.
The English summary is being prepared. Showing the original description.
Expert guidance for Fully Sharded Data Parallel training with PyTorch FSDP - parameter sharding, mixed precision, CPU offloading, FSDP2
The English summary is being prepared. Showing the original description.
Reserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.
The English summary is being prepared. Showing the original description.
Query and download public cancer imaging data from NCI Imaging Data Commons using idc-index. Use for accessing large-scale radiology (CT, MR, PET) and pathology datasets for AI training or research. No authentication required. Query by metadata, visualize in browser, check licenses.
The English summary is being prepared. Showing the original description.
Use AReaL for large-scale asynchronous LLM reinforcement learning, agentic RL, distributed post-training, inference/agent services, backend planning, and customization workflows.
The English summary is being prepared. Showing the original description.