Results for “h100”

Check live H100 or H200 GPU availability, request fixed-rate quotes, and review procurement status through the canonical Itô client. It also supports device credential revocation and explicitly authorized node checks; quote requests do not reserve capacity.

• Checking H100 or H200 availability
• Requesting a fixed-rate compute quote
• Reviewing quote and procurement status

Integrate Flash Attention into Transformer models to speed up attention, the calculation that relates input elements, while reducing GPU memory use. Covers PyTorch integration, performance measurement, output comparison, and a separate H100 optimization path.

• Optimize training with long input sequences
• Reduce memory use during long-context inference
• Measure speed and compare model outputs

Set up large language model pretraining with TorchTitan and PyTorch. Configure training across multiple GPUs, combine four ways of splitting the workload, enable Float8 calculations on H100 hardware, and manage checkpoints for saving and resuming training.

• Pretraining Llama 3.1 8B on a single node
• Launching training across multiple nodes with SLURM
• Configuring Float8 training for H100 GPUs

Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.

• Reducing GPU memory use with long sequences
• Integrating Flash Attention into PyTorch models
• Comparing runtime and outputs before and after changes

Integrate Flash Attention into transformer models to reduce processing time and GPU memory use for long sequences. Covers PyTorch integration, advanced attention options, and benchmarks that compare speed, memory use, and output accuracy.

• Speeding up transformer training on long sequences
• Reducing GPU memory use during attention
• Adding Flash Attention to a PyTorch model

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Finds the base image needed to create a custom GKE node image. It translates requirements such as GKE version, operating system, processor architecture, accelerators, and sandbox settings into a matching image name.

• Finding a base image for custom GKE nodes
• Looking up images for GPU or TPU configurations
• Matching images to COS, Ubuntu, ARM64, or X86_64

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Otimiza atenção em transformers com Flash Attention para ganho de 2-4x em velocidade e redução de 10-20x em memória. Use ao treinar/executar transformers com sequências longas (>512 tokens), ao encontrar problemas de memória GPU com atenção, ou quando precisa de inferência mais rápida. Suporta SDPA nativo do PyTorch, biblioteca flash-attn, H100 FP8 e sliding window attention.

The English summary is being prepared. Showing the original description.

Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.

The English summary is being prepared. Showing the original description.

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

The English summary is being prepared. Showing the original description.

Train language models on pairs of preferred and rejected answers without a separate reference model. The skill guides training setup, model-specific configurations, parameter tuning, and troubleshooting for unstable training or memory limits.

• Train a base model using preferred and rejected answer pairs
• Further tune an instruction-following model
• Adjust training settings for math and coding tasks

Train language models with SimPO using pairs of preferred and rejected answers. Configure preference training without a separate reference model, adapt settings for base or instruction models, and address unstable training, memory limits, or lost capabilities.

• Training a base model with preferred and rejected answers
• Further tuning an instruction model with preference data
• Adjusting training settings for math and code tasks

Explore training workflows that reduce harmful model responses using a written set of principles. The skill covers self-critique, revised training examples, and RLAIF, reinforcement learning from AI feedback, with implementation examples and hardware guidance.

• Designing principles for model safety training
• Creating training examples through self-critique and revision
• Planning reinforcement learning with AI-rated responses

Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.

• Choosing a method for training models on human preferences
• Setting up PPO training with a reward model
• Configuring GRPO training without a critic model

Guides large language model reinforcement learning with miles, covering environment setup, low-precision training, consistency between training and inference, and speculative decoding, where a smaller model proposes tokens for a larger model to verify.

• Configure reinforcement learning for large MoE models
• Investigate instability in FP8 training
• Set up speculative decoding for faster rollout generation

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Adapt PyTorch training scripts for multiple GPUs or machines with Accelerate. This skill guides code changes, configuration, and launch commands, including mixed precision, which combines numerical formats, and DeepSpeed or FSDP integration for distributed training.

• Moving single-GPU training to multiple GPUs
• Configuring mixed precision training
• Setting up DeepSpeed or FSDP

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.

• Training on preferred and rejected answer pairs
• Exploring an alternative to DPO without a reference model
• Tuning preference training for math and code tasks

Guides large language model reinforcement learning with miles, including MoE models that route work to specialized components. Covers low-precision training, matching training and inference behavior, and accelerating training response generation with draft models.

• Setting up reinforcement learning for large MoE models
• Configuring FP8 or INT4 training
• Checking consistency between training and inference

Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.

• Adapt a single-GPU training script for multiple GPUs
• Configure mixed precision to reduce training memory use
• Set up training with DeepSpeed or FSDP