62 skills · sorted by relevance
Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.
Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.
Train language models with SimPO using pairs of preferred and rejected answers. Configure preference training without a separate reference model, adapt settings for base or instruction models, and address unstable training, memory limits, or lost capabilities.
Guides large language model reinforcement learning with miles, covering environment setup, low-precision training, consistency between training and inference, and speculative decoding, where a smaller model proposes tokens for a larger model to verify.
Get guidance on fine-tuning language models with Unsloth, including LoRA and QLoRA methods for training with fewer resources. The skill points to documentation for understanding APIs, implementing training code, debugging problems, and exploring speed and memory optimizations.
Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.
Get guidance on adapting LLMs with Unsloth, including implementation, API usage, and debugging. Covers LoRA and QLoRA, methods for adjusting models with fewer trainable parameters, with a focus on training speed and memory use.
Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.
Reduce LLM weight usage through pruning, which removes selected model parameters. Explore Wanda and SparseGPT for compression without retraining, choose hardware-friendly patterns for faster inference, and compare the compressed model's accuracy with the original.
Track machine learning experiments with SwanLab by recording training settings, metrics, and media such as images and audio. Learn to compare runs, inspect training charts, and connect existing training code to local, cloud, or self-hosted dashboards.
Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.
Prepare text for language models using HuggingFace Tokenizers, which splits text into model-readable units. Train custom vocabularies, process batches, track original text positions, and configure input lengths for use with transformers.
Guides LLM compression through pruning, which removes selected model weights. Covers Wanda and SparseGPT methods without retraining, accuracy comparisons, and structured weight patterns that can accelerate inference on compatible hardware.
Prepare large text collections for AI models with fast tokenizers, which split text into model inputs. Train custom vocabularies, trace tokens to original text positions, adjust input lengths, and integrate the results with transformers.
Compress large language models to 4-bit weights to reduce GPU memory needs and accelerate text generation. Covers loading compressed models, converting your own models, choosing settings, and preparing memory-efficient fine-tuning with transformers and PEFT.
Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
The English summary is being prepared. Showing the original description.
Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
The English summary is being prepared. Showing the original description.
Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.
The English summary is being prepared. Showing the original description.
Prepare text for language models by splitting it into tokens, small units represented by numbers. The skill covers batch processing, training custom vocabularies, tracking positions in the original text, and adjusting input lengths.
Configure supervised fine-tuning for models that process images and text. Define which components remain unchanged, set LoRA adapters for updating selected model weights, choose image resolution limits, and investigate training runs whose evaluation results fail to improve.
Reduce the memory needed to run large language models with GPTQ quantization, which stores model weights at lower precision. Covers loading compressed models, quantizing your own models, choosing execution backends, and connecting to memory-efficient fine-tuning.
Reduce GPU memory use by loading language models in 8-bit or 4-bit formats with bitsandbytes and HuggingFace Transformers. Includes configuration examples, QLoRA fine-tuning of compressed models, and optimizers that reduce memory needed during training.
Build visual AI systems that detect objects or divide images into labeled regions. This skill guides dataset preparation, model selection, training, evaluation, inference optimization, and deployment using frameworks such as PyTorch and Ultralytics.
Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.