数据与 AI技能

Diagnose GPU out-of-memory errors during Megatron Bridge training and choose memory reduction settings. Covers fragmentation, LoRA input storage, recomputation, parallelism changes, and CPU offloading, explaining compatibility limits and potential tradeoffs between memory use and training throughput.

中文简介正在准备中。当前显示英语简介。

• Investigating GPU out-of-memory errors during training
• Reducing retained inputs in LoRA training
• Estimating memory needs before a large training job

Choose how to distribute tokens, the units a model processes, across MoE model experts during training. The guide compares alltoall, DeepEP, and HybridEP by GPU hardware, parallelism scale, memory needs, and optimization stage.

中文简介正在准备中。当前显示英语简介。

• Choosing a MoE dispatcher for specific GPU hardware
• Establishing an alltoall baseline for initial setup
• Comparing DeepEP and HybridEP under matching conditions

Helps tune activation recomputation in Megatron Bridge, recalculating intermediate training values to reduce GPU memory use. Guides selective or full-layer settings, checks compatibility, and compares memory peaks with training speed to assess whether a configuration completes training steps.

中文简介正在准备中。当前显示英语简介。

• Investigate training failures caused by activation memory
• Choose recomputation targets for attention or MoE models
• Compare GPU memory savings against training speed

Add support for new model architectures in NeMo AutoModel, from inspecting configuration files to implementing layers, converting saved weights, registering models, and testing numerical agreement with Hugging Face implementations.

中文简介正在准备中。当前显示英语简介。

• Adding support for a new language model
• Mapping MoE router and expert weights
• Integrating a model that handles images and text

Choose Megatron Bridge training configurations for a model, GPU allocation, and training goal. Get starting commands and parallel processing adjustments, with clear distinctions between changes that affect learning behavior, execution tuning, and benchmark-only settings.

中文简介正在准备中。当前显示英语简介。

• Choose a starting configuration for pretraining or fine-tuning
• Adapt parallel processing settings to available GPUs
• Compare training and throughput benchmark configurations

Configure and validate communication overlap in Megatron-Bridge for MoE models, which distribute work among expert networks. Check compatibility, compare training speed, and investigate regressions across standard dispatch, DeepEP, HybridEP, and delayed weight-gradient computation.

中文简介正在准备中。当前显示英语简介。

• Enable expert communication overlap in MoE training
• Check compatibility before using DeepEP or HybridEP
• Compare plain overlap and delayed weight-gradient computation

Choose how to split large-model training across GPUs in Megatron Bridge. This guide helps combine parallelism methods, estimate GPU requirements, match settings to hardware connections, and check memory use and performance after configuration changes.

中文简介正在准备中。当前显示英语简介。

• Choosing parallelism settings for a model and GPU count
• Estimating GPU requirements for MoE models
• Adjusting parallelism for long input sequences

Guide repeated NeMo-RL or NeMo-gym experiments to test hypotheses and improve training settings. Establish a baseline, compare relevant metrics, and preserve reproducible records in git and TSV logs while keeping users involved in launch decisions.

中文简介正在准备中。当前显示英语简介。

• Testing hypotheses about training settings
• Comparing accuracy, reward, or throughput
• Recording reproducible experiment histories

Helps configure training across multiple GPUs or machines in NeMo AutoModel. Guides strategy selection, YAML settings, and parallelism choices that split model computation or data, with constraints for large models, long sequences, and expert models.

中文简介正在准备中。当前显示英语简介。

• Choosing a strategy for multi-GPU training
• Configuring parallelism for large models
• Splitting long input sequences across GPUs

Search document collections with the NeMo Retriever CLI. Ingest files into a local search index or use a deployed service, then retrieve evidence for questions across PDFs, Office files, images, audio, and video.

中文简介正在准备中。当前显示英语简介。

• Find evidence across internal documents
• Build a local search index from files
• Query an existing Retriever service

Find starting configurations for training MoE models, which route work among specialized expert networks, on H100, B200, GB200, and GB300. Provides model-specific parallel layouts, communication options, and approximate performance bands to guide measured comparisons.

中文简介正在准备中。当前显示英语简介。

• Choose initial MoE training settings for specific hardware
• Compare communication backends and parallel layouts
• Estimate approximate training throughput for planning

Configure Megatron-Bridge to overlap communication with computation during distributed training. Review settings for tensor, data, and pipeline parallelism, check activation requirements, and use included tests to investigate configuration issues or throughput regressions.

中文简介正在准备中。当前显示英语简介。

• Enable communication overlap in distributed training
• Check why tensor parallel overlap is inactive
• Investigate slower training after configuration changes

Run training with mock or real data in Megatron-LM and Megatron Bridge, match their settings, and compare loss curves, which show training error over time. Includes recipes and multi-GPU examples for checking training behavior.

中文简介正在准备中。当前显示英语简介。

• Compare Megatron-LM and Bridge loss curves
• Translate MLM command-line arguments into Bridge settings
• Check training on multiple GPUs

Deploy, run, and test NVIDIA speech recognition, speech synthesis, and translation services in the cloud or on your own infrastructure. Guidance covers model selection, environment checks, custom models, and adjustments to speech processing and pronunciation.

中文简介正在准备中。当前显示英语简介。

• Deploy and test a speech recognition model
• Generate speech with Magpie
• Adjust pronunciation for specific words

Plan and tune training for vision-language models that use MoE, a mixture of expert networks, in Megatron Bridge. Compares FSDP and 3D parallel training, with guidance on memory use, batch sizes, and performance checks using realistic image inputs.

中文简介正在准备中。当前显示英语简介。

• Choosing a parallel training approach for a new model
• Investigating training failures and out-of-memory errors
• Comparing performance with realistic image inputs

Enable Megatron FSDP in Megatron-Bridge to distribute model training state across GPUs. Follow configuration and checkpoint requirements, investigate memory failures or performance regressions, and verify the setup with an existing two-GPU functional test.

中文简介正在准备中。当前显示英语简介。

• Switch from DDP to Megatron FSDP
• Investigate memory failures after FSDP configuration changes
• Check checkpoint formats and incompatible settings

Configure and validate CUDA graphs in Megatron Bridge training. Choose whole-iteration or module-level capture, which records GPU operations for replay, then check compatibility constraints and compare runtime and memory use against ordinary execution.

中文简介正在准备中。当前显示英语简介。

• Choose whole-iteration or module-level CUDA graph capture
• Configure graph scopes for dense or MoE training
• Investigate crashes after CUDA graph configuration changes

Export existing medical AI inference results to MLflow, a system for recording machine learning runs. Preview sanitized metadata, parameters, reproducibility details, and engineering quality metrics before export, with optional image artifacts under explicit upload policies.

中文简介正在准备中。当前显示英语简介。

• Recording completed inference runs in MLflow
• Keeping generation parameters and reproducibility details
• Previewing export contents without contacting MLflow

Measure language model response times and generation speed on Jetson using wrappers for vLLM, llama.cpp, or Ollama. Produces structured JSON results to assess models, power modes, runtime settings, and performance changes after tuning.

中文简介正在准备中。当前显示英语简介。

• Measure response times for a deployed model
• Compare performance before and after tuning
• Evaluate models and power modes under controlled conditions

Choose software for serving language or vision-language models on NVIDIA Jetson using a current memory audit. Get specific launch flags for memory limits, context length, and concurrency to address memory shortages or compare lower-memory alternatives.

中文简介正在准备中。当前显示英语简介。

• Choosing serving software for a Jetson memory budget
• Adjusting vLLM flags after out-of-memory errors
• Considering a switch from vLLM to llama.cpp

Launch distributed Megatron-LM training on a SLURM cluster with guidance on job scripts, environment variables, GPU settings, and containers. Monitor jobs and investigate failures across worker processes to identify the initial cause.

中文简介正在准备中。当前显示英语简介。

• Write an sbatch script for distributed training
• Configure training across multiple GPU nodes
• Check CUDA settings for the parallelism mode

Configure speculative decoding on Jetson vLLM servers, letting a smaller model propose tokens for the main model to verify. Adjust memory allocation and benchmark before and after to assess generation speed under low concurrency.

中文简介正在准备中。当前显示英语简介。

• Reducing slow token generation on Thor or AGX Orin
• Choosing EAGLE-3 or a small draft model
• Comparing generation speed before and after configuration

Train and evaluate robot policies, the models that choose actions, using RSL-RL or RLinf in healthcare simulations. Export trained artifacts and check that they work through the normal Workflow runner.

中文简介正在准备中。当前显示英语简介。

• Train an ultrasound probe reach policy
• Post-train a Trocar policy from a local GR00T checkpoint
• Evaluate and export an existing training checkpoint

Provides instructions for serving language and vision-language models on Jetson with vLLM or SGLang. Selects a runtime for the device and JetPack version, supplies launch commands, and explains how to verify the OpenAI-compatible API.

中文简介正在准备中。当前显示英语简介。

• Starting a language-model API on Jetson
• Connecting Open WebUI or an app to a model server
• Serving a vision-language model with vLLM