Results for “rlhf”

Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.

• Choosing a method for training models on human preferences
• Setting up PPO training with a reward model
• Configuring GRPO training without a critic model

Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.

• Choosing a training method for human preference alignment
• Setting up distributed training across multiple GPUs
• Training a reward model before PPO

Guide language model training with TRL using example answers, preferred response pairs, or reward scores. The skill covers data preparation, method selection, reward model training, evaluation, and adjustments for memory limits or unstable training.

• Train a language model to follow instructions using example answers
• Train on preferred and rejected responses with DPO
• Build a reward model and run an RLHF training pipeline

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Framework RLHF de alta performance com aceleração Ray+vLLM. Use para treinamento PPO, GRPO, RLOO, DPO de modelos grandes (7B-70B+). Construído em Ray, vLLM, ZeRO-3. 2× mais rápido que DeepSpeedChat com arquitetura distribuída e compartilhamento de recursos GPU.

The English summary is being prepared. Showing the original description.

High-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.

The English summary is being prepared. Showing the original description.

Set up reinforcement learning, which trains models using rewards, with verl. Follow workflows for preparing data, configuring GRPO or PPO training, scaling across GPUs, and addressing memory errors or unstable training.

• Training a math reasoning model with GRPO
• Configuring PPO training with a critic model
• Scaling large model training across multiple nodes

Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.

• Train math reasoning models with GRPO and answer-based rewards
• Configure PPO training with a separate value-estimation model
• Run large-model training across multiple GPU nodes

Prepare high-quality datasets for LLM fine-tuning with filtering, deduplication, augmentation, and RLHF data formatting. Activate on: fine-tuning data, training data curation, RLHF dataset, data quality filtering, SFT dataset. NOT for: model training infrastructure (ai-engineer), prompt engineering without fine-tuning (prompt-engineer).

The English summary is being prepared. Showing the original description.

Set up and run the full RLHF pipeline (SFT, reward model training, RL from reward model) using the Tinker API. Use when the user wants to do RLHF, train a reward model, or run the full preference-based RL pipeline.

The English summary is being prepared. Showing the original description.

Prepare high-quality datasets for LLM fine-tuning with filtering, deduplication, augmentation, and RLHF data formatting. Activate on: fine-tuning data, training data curation, RLHF dataset, data quality filtering, SFT dataset. NOT for: model training infrastructure (ai-engineer), prompt engineering without fine-tuning (prompt-engineer).

The English summary is being prepared. Showing the original description.

Design rewards for reinforcement learning, where feedback guides model behavior. Choose how to score outcomes and intermediate actions, check for scoring loopholes, and use independent tests to detect incomplete work presented as finished.

• Designing rewards for reinforcement learning
• Investigating scoring loopholes and early completion
• Choosing outcome rewards or intermediate feedback

Fine-tune language models with TRL using GRPO, a method that compares scored responses to the same prompt. Covers training data, scoring functions, GPU-conscious settings, and monitoring for tasks with measurable correctness or required output formats.

• Train models to follow JSON or XML output formats
• Reward correct answers in math or coding tasks
• Balance correctness, format, and style during training

Prepare for Anthropic-specific technical interviews covering Constitutional AI, RLHF, interpretability, scaling laws, MCP, agentic systems, and AI safety. Activate on "anthropic interview", "constitutional AI prep", "alignment interview", "safety interview", "RLHF opinion", "interpretability prep", "anthropic technical". NOT for general ML interview prep, system design interviews, coding challenges, or behavioral interview practice.

The English summary is being prepared. Showing the original description.

Ajuste fino de LLMs usando reinforcement learning com TRL - SFT para ajuste de instruções, DPO para alinhamento de preferências, PPO/GRPO para otimização de recompensas e treinamento de modelo de recompensas. Use quando precisar de RLHF, alinhar modelo com preferências ou treinar a partir de feedback humano. Funciona com HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Fornece orientação para treinar LLMs com aprendizado por reforço usando verl (Volcano Engine RL). Use ao implementar RLHF, GRPO, PPO ou outros algoritmos de RL para pós-treinamento de LLMs em escala com backends de infraestrutura flexíveis.

The English summary is being prepared. Showing the original description.

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Prepare for Anthropic-specific technical interviews covering Constitutional AI, RLHF, interpretability, scaling laws, MCP, agentic systems, and AI safety. Activate on "anthropic interview", "constitutional AI prep", "alignment interview", "safety interview", "RLHF opinion", "interpretability prep", "anthropic technical". NOT for general ML interview prep, system design interviews, coding challenges, or behavioral interview practice.

The English summary is being prepared. Showing the original description.

Guides language model training with GRPO in TRL, comparing several responses and rewarding better results. Covers dataset preparation, scoring rules for correctness and format, training configuration, monitoring, and saving models for later use.

• Training models to follow JSON or XML formats
• Designing rewards for math and coding tasks
• Adjusting training to available GPU memory

Create a Chinese interview preparation guide for a focused machine learning or LLM topic, with formula derivations, PyTorch implementations, comparison tables, and 25 questions across three difficulty levels. The workflow reviews the content and renders it as HTML.

• Preparing for an interview on a specific machine learning topic
• Studying formula derivations alongside PyTorch implementations
• Organizing interview questions by difficulty

Adapt large language models to specific tasks through fine-tuning, or additional training. Prepares and validates datasets, configures LoRA or QLoRA to train a small subset of parameters, and supplies training settings, evaluation scripts, and deployment optimization guidance.

• Preparing and validating JSONL training data
• Choosing LoRA or QLoRA for available GPU memory
• Configuring training and monitoring overfitting

Plan, configure, and chain repo-native Nemotron customization steps into single-step or multi-step pipelines: curation, translation, SFT/PEFT (AutoModel or Megatron-Bridge), pretraining/CPT, RL alignment (DPO/RLVR/GRPO/RLHF), BYOB/MCQ benchmarks, checkpoint conversion, ModelOpt optimization, env profiles, and evaluation of trained checkpoints or existing/hosted endpoints. Use when a request names a Nemotron step or workflow, or asks to clean, translate, train, fine-tune, align, convert, optimize, evaluate, or compose these into a pipeline. Do NOT use for frontend/dashboard/visualization work, generic ML advice, billing/access, or non-Nemotron coding tasks.

The English summary is being prepared. Showing the original description.

This skill should be used when picking or diagnosing a training move (SFT, LoRA, DPO/KTO/ORPO, RFT, GRPO/PPO/RLOO, RLHF), or when the user mentions fine-tuning, post-training, training recipe, reward design, or weight updates. Decision tree by reward shape, smoke-run gate, three failure diagnostics, five false-progress patterns. Provider recipes and I/O contract in references/.

The English summary is being prepared. Showing the original description.