'expert parallelism' 검색 결과

Build and train Mixture of Experts models, which activate selected specialist networks for each input. Use DeepSpeed or HuggingFace workflows to configure routing, balance workloads, distribute experts across GPUs, and optimize inference.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing MoE models for a limited compute budget
• Implementing Mixtral-style expert routing
• Adjusting expert workloads and training stability

Train Mixture of Experts models, which activate selected specialist networks for each input. The skill guides architecture, input routing, workload balancing, and GPU distribution using DeepSpeed or HuggingFace, with examples for training configuration and inference optimization.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing a model that activates only selected expert networks
• Balancing workloads across expert networks
• Distributing experts across multiple GPUs

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Treinar modelos de Mixture of Experts (MoE) usando DeepSpeed ou HuggingFace. Use ao treinar modelos em larga escala com computação limitada (redução de 5× em custos vs modelos densos), implementar arquiteturas esparsas como Mixtral 8x7B ou DeepSeek-V3, ou escalar capacidade de modelo sem aumento proporcional de computação. Cobre arquiteturas MoE, mecanismos de roteamento, balanceamento de carga, paralelismo de especialistas e otimização de inferência.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Expert en concurrence (threads, async/await, mutex, channels, race conditions)

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.