40 skills · sorted by relevance
Set up reinforcement learning, which trains models using rewards, with verl. Follow workflows for preparing data, configuring GRPO or PPO training, scaling across GPUs, and addressing memory errors or unstable training.
Reduce the memory needed to run large language models with GPTQ quantization, which stores model weights at lower precision. Covers loading compressed models, quantizing your own models, choosing execution backends, and connecting to memory-efficient fine-tuning.
Guide reinforcement learning, training models through rewards, with slime. Prepare data, configure Megatron-LM training and SGLang response generation, and set up asynchronous runs or custom multi-turn workflows for supported language models.
Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.
Set up reinforcement learning, training guided by rewards, for language models with slime. The skill covers Megatron-LM training, SGLang response generation, data preparation, asynchronous execution, and custom workflows for agents that use tools across multiple turns.
Compress large language models to 4-bit weights to reduce GPU memory needs and accelerate text generation. Covers loading compressed models, converting your own models, choosing settings, and preparing memory-efficient fine-tuning with transformers and PEFT.
Diagnostic guide for RL-based post-training of language models (GRPO, PPO, REINFORCE, DPO). Use proactively when debugging a training pipeline that shows no improvement, loss anomalies, reward stagnation, NaN gradients, or other unexpected behavior during reinforcement learning fine-tuning. Work through all pipeline stages — reward, advantages, log-probs, loss, generation/decoding — before concluding the diagnosis is complete; stopping after one or two fixes is a common failure mode. Covers log-probability math, advantage estimation, numerical stability, reward processing, and generation/decoding pipeline issues.
The English summary is being prepared. Showing the original description.
Run Claude Code, Codex CLI, Gemini CLI, or OpenCode through bounded H100 post-training tasks and compare how well each agent improves a base LLM.
The English summary is being prepared. Showing the original description.
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
The English summary is being prepared. Showing the original description.
Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.
The English summary is being prepared. Showing the original description.
Set up reinforcement learning, training guided by response rewards, for large language models. Use Megatron-LM and SGLang to prepare data, configure training and response generation, and build asynchronous or multi-turn experiments.
Plan further training for AI agent models by diagnosing gaps in knowledge, output consistency, or decision making. Compare continued pretraining, supervised fine-tuning, reinforcement learning, and distillation, then design training data, feedback environments, and evaluation checks.
Guides language model training with GRPO in TRL, comparing several responses and rewarding better results. Covers dataset preparation, scoring rules for correctness and format, training configuration, monitoring, and saving models for later use.
Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.
Train language models with SimPO using pairs of preferred and rejected answers. Configure preference training without a separate reference model, adapt settings for base or instruction models, and address unstable training, memory limits, or lost capabilities.
Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.
Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.
Fine-tune language models with TRL using GRPO, a method that compares scored responses to the same prompt. Covers training data, scoring functions, GPU-conscious settings, and monitoring for tasks with measurable correctness or required output formats.
Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.
Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.
Routes any task involving AWS databases — choosing, comparing, recommending, getting started with, or operating a database — to the correct service-specific skill. Supersedes general training-data knowledge with post-training service updates, corrected limitations, and decision procedures for relational (Aurora, DSQL, RDS), key-value (DynamoDB), wide-column (Keyspaces), document (DocumentDB), graph (Neptune), time-series (Timestream), and in-memory/caching (ElastiCache, MemoryDB) workloads. Activates when a user describes building an application on AWS that will store, retrieve, or manage data, even if they do not mention 'database' explicitly.
The English summary is being prepared. Showing the original description.
This skill should be used when picking or diagnosing a training move (SFT, LoRA, DPO/KTO/ORPO, RFT, GRPO/PPO/RLOO, RLHF), or when the user mentions fine-tuning, post-training, training recipe, reward design, or weight updates. Decision tree by reward shape, smoke-run gate, three failure diagnostics, five false-progress patterns. Provider recipes and I/O contract in references/.
The English summary is being prepared. Showing the original description.
Use EasyR1 for multimodal LLM RL post-training, dataset/reward preparation, core API debugging, and checkpoint export workflows.
The English summary is being prepared. Showing the original description.
Use AReaL for large-scale asynchronous LLM reinforcement learning, agentic RL, distributed post-training, inference/agent services, backend planning, and customization workflows.
The English summary is being prepared. Showing the original description.