“deepspeed”的搜索结果

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

中文简介正在准备中。当前显示英语简介。

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

中文简介正在准备中。当前显示英语简介。

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Adapt existing PyTorch training scripts to use multiple GPUs or machines with limited code changes. The skill guides configuration, launch commands, mixed precision, which uses lower-precision numbers, and DeepSpeed or FSDP settings for distributing training work and memory.

中文简介正在准备中。当前显示英语简介。

• Move a single-GPU training script to multiple GPUs
• Configure mixed precision training
• Set up DeepSpeed or FSDP to reduce memory use

Adapt PyTorch training scripts for multiple GPUs or machines with Accelerate. This skill guides code changes, configuration, and launch commands, including mixed precision, which combines numerical formats, and DeepSpeed or FSDP integration for distributed training.

中文简介正在准备中。当前显示英语简介。

• Moving single-GPU training to multiple GPUs
• Configuring mixed precision training
• Setting up DeepSpeed or FSDP

Train Mixture of Experts models, which activate selected specialist networks for each input. The skill guides architecture, input routing, workload balancing, and GPU distribution using DeepSpeed or HuggingFace, with examples for training configuration and inference optimization.

中文简介正在准备中。当前显示英语简介。

• Designing a model that activates only selected expert networks
• Balancing workloads across expert networks
• Distributing experts across multiple GPUs

Build and train Mixture of Experts models, which activate selected specialist networks for each input. Use DeepSpeed or HuggingFace workflows to configure routing, balance workloads, distribute experts across GPUs, and optimize inference.

中文简介正在准备中。当前显示英语简介。

• Designing MoE models for a limited compute budget
• Implementing Mixtral-style expert routing
• Adjusting expert workloads and training stability

Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.

中文简介正在准备中。当前显示英语简介。

• Adapt a single-GPU training script for multiple GPUs
• Configure mixed precision to reduce training memory use
• Set up training with DeepSpeed or FSDP

Organize PyTorch training code with LightningModule and Trainer. This skill guides training, validation, logging, model checkpoint saving, early stopping when progress stalls, and configuration for running the same model across multiple GPUs.

中文简介正在准备中。当前显示英语简介。

• Organize existing PyTorch training code
• Add validation and metric logging
• Save the best models and stop stalled training

Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention

中文简介正在准备中。当前显示原文说明。

Organize PyTorch models and data pipelines with PyTorch Lightning, automate training, save checkpoints, and track experiments. Includes templates and guidance for configuring training across multiple GPUs or TPUs.

中文简介正在准备中。当前显示英语简介。

• Restructure PyTorch training code into LightningModules
• Build reusable data preparation and loading pipelines
• Configure checkpoints, early stopping, and experiment logs

Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.

中文简介正在准备中。当前显示英语简介。

• Choosing a method for training models on human preferences
• Setting up PPO training with a reward model
• Configuring GRPO training without a critic model

Get guidance on adapting large language models with Axolotl. Covers YAML configuration files, LoRA and QLoRA methods for training fewer parameters, distributed training settings, dataset formats, and debugging training code.

中文简介正在准备中。当前显示英语简介。

• Write YAML settings for model fine-tuning
• Explore training with LoRA or QLoRA
• Configure training across multiple GPUs

Organize PyTorch training code with LightningModule and Trainer. Follow examples for validation, logging, model saving, and distributed training across multiple GPUs, with callbacks that add monitoring or stop training when progress stalls.

中文简介正在准备中。当前显示英语简介。

• Converting an existing PyTorch training loop
• Adding validation, logs, and model checkpoints
• Configuring training across multiple GPUs

Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.

中文简介正在准备中。当前显示英语简介。

• Choosing a training method for human preference alignment
• Setting up distributed training across multiple GPUs
• Training a reward model before PPO

Guides LLM fine-tuning with Axolotl, from YAML configuration and dataset formats to distributed training across GPUs. Covers LoRA and QLoRA, methods that train on preferences, multimodal support, and troubleshooting using documentation references and code examples.

中文简介正在准备中。当前显示英语简介。

• Prepare YAML settings for LLM fine-tuning
• Explore LoRA, QLoRA, and preference training options
• Check dataset formats and input handling

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

中文简介正在准备中。当前显示原文说明。

Use DeepSpeed for distributed training, inference acceleration, ZeRO configuration, parallelism/MoE design, profiling, autotuning, and operational diagnostics.

中文简介正在准备中。当前显示原文说明。

API de treinamento distribuído mais simples. 4 linhas para adicionar suporte distribuído a qualquer script PyTorch. API unificada para DeepSpeed/FSDP/Megatron/DDP. Posicionamento automático de device, precisão mista (FP16/BF16/FP8). Config interativo, comando de launch único. Padrão do ecossistema HuggingFace.

中文简介正在准备中。当前显示原文说明。

Treinar modelos de Mixture of Experts (MoE) usando DeepSpeed ou HuggingFace. Use ao treinar modelos em larga escala com computação limitada (redução de 5× em custos vs modelos densos), implementar arquiteturas esparsas como Mixtral 8x7B ou DeepSeek-V3, ou escalar capacidade de modelo sem aumento proporcional de computação. Cobre arquiteturas MoE, mecanismos de roteamento, balanceamento de carga, paralelismo de especialistas e otimização de inferência.

中文简介正在准备中。当前显示原文说明。

Framework PyTorch de alto nível com classe Trainer, treinamento distribuído automático (DDP/FSDP/DeepSpeed), sistema de callbacks e boilerplate mínimo. Escala de laptop para supercomputador com o mesmo código. Use quando quiser loops de treinamento limpos com boas práticas integradas.

中文简介正在准备中。当前显示原文说明。

High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.

中文简介正在准备中。当前显示原文说明。

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

中文简介正在准备中。当前显示原文说明。

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

中文简介正在准备中。当前显示原文说明。

High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.

中文简介正在准备中。当前显示原文说明。