139 skills · sorted by relevance
Compress large language models to 4-bit weights to reduce GPU memory use and accelerate inference. Covers loading compressed models, configuring quantization, and using Transformers or vLLM, with guidance on hardware requirements and backend choices.
Train smaller language models using knowledge distillation, where a larger model supplies examples or prediction targets. This skill covers training losses, generated training data, parameter choices, and output comparisons to support model compression and specialized student models.
Reduce GPU memory use by loading language models in 8-bit or 4-bit formats with bitsandbytes and HuggingFace Transformers. Includes configuration examples, QLoRA fine-tuning of compressed models, and optimizers that reduce memory needed during training.
Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.
Build and train Mixture of Experts models, which activate selected specialist networks for each input. Use DeepSpeed or HuggingFace workflows to configure routing, balance workloads, distribute experts across GPUs, and optimize inference.
Guide knowledge distillation, where a smaller language model learns from a larger teacher. Covers training from teacher responses or prediction scores, adjusting training settings, and comparing outputs to support model compression and specialized training.
Compress large language models to 4-bit weights to reduce GPU memory needs and accelerate text generation. Covers loading compressed models, converting your own models, choosing settings, and preparing memory-efficient fine-tuning with transformers and PEFT.
Guides large language model reinforcement learning with miles, including MoE models that route work to specialized components. Covers low-precision training, matching training and inference behavior, and accelerating training response generation with draft models.
Compress large language model weights to four bits with AWQ, a method that uses model activation patterns to limit quality loss. Follow workflows for quantization, loading compressed models, and serving them with vLLM using less GPU memory.
Guides large language model reinforcement learning with miles, covering environment setup, low-precision training, consistency between training and inference, and speculative decoding, where a smaller model proposes tokens for a larger model to verify.
Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.
Reduce the memory needed to run large language models with GPTQ quantization, which stores model weights at lower precision. Covers loading compressed models, quantizing your own models, choosing execution backends, and connecting to memory-efficient fine-tuning.
Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.
Train Mixture of Experts models, which activate selected specialist networks for each input. The skill guides architecture, input routing, workload balancing, and GPU distribution using DeepSpeed or HuggingFace, with examples for training configuration and inference optimization.
Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.
Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.
Adapt large language models to specific tasks through fine-tuning, or additional training. Prepares and validates datasets, configures LoRA or QLoRA to train a small subset of parameters, and supplies training settings, evaluation scripts, and deployment optimization guidance.
Set up vLLM to serve large language models through an OpenAI-compatible API or process prompts in batches. The skill covers deployment, performance monitoring, and quantization, which reduces model memory requirements.
Evaluate large language models on shared tests of knowledge, reasoning, and code generation with lm-evaluation-harness. Save scores and individual predictions, compare models in tables, and track performance across saved stages of model training.
Implement and debug large-model training with PyTorch FSDP, which splits training state across devices. Consult documentation and code patterns for sharding choices, mixed precision, and checkpoints when building or reviewing distributed training code.
Coordinate large research or generation tasks across a model team. An orchestrator plans and checks the work, workers execute subtasks in parallel, and an advisor reviews the plan and draft before the results are combined.
Build graph neural networks, models that learn from connected nodes, with PyTorch Geometric. Supports node and graph classification, link prediction, custom datasets, and training approaches for large graphs or graphs containing multiple types of nodes and connections.
Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.
Process large email, document, and transcript archives by routing items to AI models according to their importance. Rule-based privacy checks run before model calls, while extracted information becomes linked knowledge pages with progress checkpoints.