82 skills · sorted by relevance
Set up model training with torchforge for reinforcement learning, where rewards guide improvement. Covers training configurations, custom reward and loss functions, and distributed execution with Monarch and TorchTitan, keeping algorithm experiments separate from infrastructure concerns.
Configure machine learning training and batch jobs across multiple clouds with SkyPilot. The skill guides GPU selection, lower-cost cloud and region choices, and automatic recovery when discounted spot instances are interrupted.
Get guidance on adapting large language models with Axolotl. Covers YAML configuration files, LoRA and QLoRA methods for training fewer parameters, distributed training settings, dataset formats, and debugging training code.
Manage GPU compute jobs on Qizhi through a command-line tool. Find available nodes, submit distributed training runs, launch batches with different parameter combinations, and inspect, monitor, or stop jobs across workspaces.
Configure machine learning training and batch jobs across multiple clouds with SkyPilot. The skill guides GPU selection, lower-cost cloud and region choices, and automatic recovery when discounted spot instances are interrupted.
Guide for selecting and configuring distributed training strategies in NeMo AutoModel, including FSDP2, Megatron FSDP, DDP, and parallelism settings.
The English summary is being prepared. Showing the original description.
Distributed machine learning, data mining, and iterative HPC with Exasol. Covers end-to-end ML pipelines (DISTRIBUTE BY + SET scripts + BucketFS), per-entity federated training with partial_fit and ctx.reset(), batch inference, map-reduce ensemble training, distributed ensemble and SON algorithm for frequent itemset mining (Apriori, FP-Growth, association rules, market-basket analysis), Lua execute script orchestration for iterative algorithms (k-means, SGD, gradient descent), scikit-learn model training, parallel hyperparameter search, per-entity forecasting, anomaly detection, model lifecycle in BucketFS (pickle/joblib/ONNX versioning), GPU acceleration via CUDA SLCs (PyTorch/TensorFlow/RAPIDS), and ML-specific performance tuning (skew, OOM, multi-pass chunking).
The English summary is being prepared. Showing the original description.
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.
The English summary is being prepared. Showing the original description.
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
The English summary is being prepared. Showing the original description.
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.
The English summary is being prepared. Showing the original description.
Set up GPU servers on Lambda Labs for machine learning training and inference. Follow workflows for SSH access, persistent storage, Python API operations, and distributed training that spreads work across multiple GPUs or servers.
Guides reinforcement learning, where models learn from rewards, using torchforge and PyTorch. Covers reward design, custom loss functions, and distributed training so researchers can experiment with algorithms while keeping infrastructure concerns separate.
Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor, scatter_tensor, domain parallelism, sequence/spatial sharding, ring attention, DeviceMesh + DDP/FSDP2 hybrid parallelism, or physicsnemo.domain_parallel. Do NOT use for generic PyTorch DDP/FSDP setup without domain parallelism, picking a PhysicsNeMo model or example (use physicsnemo-discover), or non-distributed training questions.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Implements federated learning architecture patterns for GDPR compliance. Covers secure aggregation protocols, differential privacy integration, communication protocols, and privacy-by-design distributed ML training. Keywords: federated learning, distributed training, secure aggregation, differential privacy, privacy-preserving ML.
The English summary is being prepared. Showing the original description.
Fornece treinamento distribuído de LLM nativo do PyTorch usando torchtitan com paralelismo 4D (FSDP2, TP, PP, CP). Use para fazer pré-treinamento de Llama 3.1, DeepSeek V3 ou modelos customizados em escala de 8 a 512+ GPUs com Float8, torch.compile e checkpoint distribuído.
The English summary is being prepared. Showing the original description.
High-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.
The English summary is being prepared. Showing the original description.
Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Prepare LLM training datasets with NeMo Curator by removing duplicates, filtering low-quality content, and redacting personal information. The skill also covers image, video, and audio curation, including GPU processing and distributed workflows.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.
Run batch processing, scientific simulations, and machine learning training on Google Kubernetes Engine. Configure parallel jobs, queues that manage resource access, and distributed computation, with guidance on scaling idle capacity down and recovering from interruptions.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.