Data analysis, machine learning, LLM apps, and prompt design (9.2K skills)
Choose how to distribute tokens, the units a model processes, across MoE model experts during training. The guide compares alltoall, DeepEP, and HybridEP by GPU hardware, parallelism scale, memory needs, and optimization stage.
Improve MoE training throughput in Megatron Bridge. MoE models divide work among expert networks; this skill identifies memory, communication, and compute bottlenecks, guides controlled configuration comparisons, and validates speed alongside training health.
Guides hierarchical context parallelism in Megatron-Bridge, which splits input sequences across nested GPU groups. Covers configuration, implementation locations, constraints, and log checks to help enable the feature and investigate memory errors or performance regressions.
Guide repeated NeMo-RL or NeMo-gym experiments to test hypotheses and improve training settings. Establish a baseline, compare relevant metrics, and preserve reproducible records in git and TSV logs while keeping users involved in launch decisions.
Plan and troubleshoot Nemotron retrieval recipes for embeddings, which turn text into searchable vectors, and reranking, which reorders search results. Review configurations, prepare training and evaluation commands, and check export or deployment requirements.
Configure CPU offloading in Megatron Bridge to reduce GPU memory use by moving intermediate training data or optimizer states to CPU memory. Check compatibility, adjust the offloaded fraction, and investigate memory errors or slowdowns.
Helps configure training across multiple GPUs or machines in NeMo AutoModel. Guides strategy selection, YAML settings, and parallelism choices that split model computation or data, with constraints for large models, long sequences, and expert models.
Choose how to split large-model training across GPUs in Megatron Bridge. This guide helps combine parallelism methods, estimate GPU requirements, match settings to hardware connections, and check memory use and performance after configuration changes.
Helps tune activation recomputation in Megatron Bridge, recalculating intermediate training values to reduce GPU memory use. Guides selective or full-layer settings, checks compatibility, and compares memory peaks with training speed to assess whether a configuration completes training steps.
Create and modify NeMo AutoModel training and evaluation recipes. Get guidance on YAML configuration, functions that build training components, execution flow, validation, and checkpoint saving and resuming, with minimal examples and local checks.
Search documents through NeMo Retriever MCP and use retrieved passages to support answers. Add missing documents when server access is confirmed, then search them while preserving available document identifiers and page numbers for citations.
Plan long-sequence training for MoE models, which divide work among expert networks, in Megatron Bridge. Use experiment-based guidance to balance parallelism, selective recalculation, and routing choices when investigating memory shortages or reduced training throughput.
Configure and validate communication overlap in Megatron-Bridge for MoE models, which distribute work among expert networks. Check compatibility, compare training speed, and investigate regressions across standard dispatch, DeepEP, HybridEP, and delayed weight-gradient computation.
Run training with mock or real data in Megatron-LM and Megatron Bridge, match their settings, and compare loss curves, which show training error over time. Includes recipes and multi-GPU examples for checking training behavior.
Configure Megatron-Bridge to overlap communication with computation during distributed training. Review settings for tensor, data, and pipeline parallelism, check activation requirements, and use included tests to investigate configuration issues or throughput regressions.
Search document collections with the NeMo Retriever CLI. Ingest files into a local search index or use a deployed service, then retrieve evidence for questions across PDFs, Office files, images, audio, and video.
Configure NeMo AutoModel training launches on a local node, a Slurm cluster scheduler, or cloud resources through SkyPilot. Prepare launch settings for GPUs, containers, environment variables, multi-node communication, and optional performance profiling.
Enable Megatron FSDP in Megatron-Bridge to distribute model training state across GPUs. Follow configuration and checkpoint requirements, investigate memory failures or performance regressions, and verify the setup with an existing two-GPU functional test.
Launch distributed Megatron-LM training on a SLURM cluster with guidance on job scripts, environment variables, GPU settings, and containers. Monitor jobs and investigate failures across worker processes to identify the initial cause.
Choose software for serving language or vision-language models on NVIDIA Jetson using a current memory audit. Get specific launch flags for memory limits, context length, and concurrency to address memory shortages or compare lower-memory alternatives.
Measure language model response times and generation speed on Jetson using wrappers for vLLM, llama.cpp, or Ollama. Produces structured JSON results to assess models, power modes, runtime settings, and performance changes after tuning.
Configure speculative decoding on Jetson vLLM servers, letting a smaller model propose tokens for the main model to verify. Adjust memory allocation and benchmark before and after to assess generation speed under low concurrency.
Train and evaluate robot policies, the models that choose actions, using RSL-RL or RLinf in healthcare simulations. Export trained artifacts and check that they work through the normal Workflow runner.
Provides instructions for serving language and vision-language models on Jetson with vLLM or SGLang. Selects a runtime for the device and JetPack version, supplies launch commands, and explains how to verify the OpenAI-compatible API.