Results for “llm alignment”

Build safety checks for LLM prompts and responses using Meta's LlamaGuard. This skill covers content classification, batch moderation, serving a moderation API, and connecting checks to NeMo Guardrails.

• Screening user prompts before sending them to an LLM
• Checking generated responses before displaying them
• Building an API for content safety checks

Build safety checks for LLM prompts and responses using Meta's LlamaGuard moderation model. Classify potentially harmful content, block flagged messages, serve checks through an API, and integrate them with NeMo Guardrails.

• Checking prompts before sending them to an LLM
• Filtering responses before displaying them
• Building a moderation API

Train language models with SimPO using pairs of preferred and rejected answers. Configure preference training without a separate reference model, adapt settings for base or instruction models, and address unstable training, memory limits, or lost capabilities.

• Training a base model with preferred and rejected answers
• Further tuning an instruction model with preference data
• Adjusting training settings for math and code tasks

Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.

• Training on preferred and rejected answer pairs
• Exploring an alternative to DPO without a reference model
• Tuning preference training for math and code tasks

Use Meta's Prompt Guard to detect malicious instructions in user prompts and external data. This skill guides input filtering, batch checks for retrieved documents, detection threshold adjustments, and splitting long text for inspection.

• Detecting attempts to override an LLM's instructions
• Checking API responses for embedded malicious instructions
• Batch screening documents retrieved for RAG

Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.

• Choosing a training method for human preference alignment
• Setting up distributed training across multiple GPUs
• Training a reward model before PPO

Add safety checks to LLM applications with NeMo Guardrails. This skill guides input and output validation, detection of attempts to bypass safety rules, personal information masking, and checks for unsupported or inaccurate answers.

• Detect attempts to bypass an AI app's safety rules
• Mask personal information before processing
• Add fact-checking to generated answers

Configure safety checks for LLM applications with NeMo Guardrails. Covers attempts to bypass safety rules, input and output validation, masking personal information, and checking factual claims through programmable rules and integrations.

• Detecting instructions that try to bypass safety rules
• Masking personal information before processing
• Adding factual checks to generated answers

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

The English summary is being prepared. Showing the original description.

Train language models on pairs of preferred and rejected answers without a separate reference model. The skill guides training setup, model-specific configurations, parameter tuning, and troubleshooting for unstable training or memory limits.

• Train a base model using preferred and rejected answer pairs
• Further tune an instruction-following model
• Adjust training settings for math and coding tasks

Systematically bypass LLM safety filters and content moderation systems using advanced jailbreaking techniques. Use this skill when testing AI systems for safety alignment robustness, evaluating content moderation effectiveness, or conducting authorized AI red team assessments. Covers role-play attacks, few-shot manipulation, encoding bypass, multi-modal exploitation, and automated jailbreak discovery methods.

The English summary is being prepared. Showing the original description.

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Execute advanced LLM Jailbreaking techniques using roleplay, nested environments (virtual machines), and complex personas to completely bypass safety constraints and ethical alignments embedded in AI models.

The English summary is being prepared. Showing the original description.

Simple Preference Optimization para alinhamento de LLMs. Alternativa sem modelo de referência ao DPO com melhor desempenho (+6.4 pontos no AlpacaEval 2.0). Sem modelo de referência necessário, mais eficiente que DPO. Use para alinhamento de preferências quando quer treinamento mais simples e rápido que DPO/PPO.

The English summary is being prepared. Showing the original description.

Comprehensive LLM security testing: bias detection, data leakage, prompt injection, jailbreak, alignment testing, adversarial prompts

The English summary is being prepared. Showing the original description.

Guides large language model reinforcement learning with miles, including MoE models that route work to specialized components. Covers low-precision training, matching training and inference behavior, and accelerating training response generation with draft models.

• Setting up reinforcement learning for large MoE models
• Configuring FP8 or INT4 training
• Checking consistency between training and inference

Use when the user wants to design, redesign, shape, critique, audit, polish, clarify, distill, harden, optimize, adapt, animate, colorize, extract, or otherwise improve a frontend interface. Covers websites, landing pages, dashboards, product UI, app shells, components, forms, settings, onboarding, and empty states. Handles UX review, visual hierarchy, information architecture, cognitive load, accessibility, performance, responsive behavior, theming, anti-patterns, typography, fonts, spacing, layout, alignment, color, motion, micro-interactions, UX copy, error states, edge cases, i18n, and reusable design systems or tokens. Also use for bland designs that need to become bolder or more delightful, loud designs that should become quieter, live browser iteration on UI elements, or ambitious visual effects that should feel technically extraordinary. Not for backend-only or non-UI tasks.

The English summary is being prepared. Showing the original description.

Optimize content for AI-generated responses and LLM citations (ChatGPT, Perplexity, Google AI Overview, Claude, Gemini). Use when the user mentions 'GEO', 'AEO', 'AI SEO', 'LLM optimization', 'citation rate', 'AI visibility', 'optimize for ChatGPT', 'roundup pages', or wants to audit pages for AI discoverability. Includes terminology alignment, FAQ schemas, and community signal strategies.

The English summary is being prepared. Showing the original description.

Comprehensive LLM security testing prompts for bias detection, data leakage, alignment testing, and adversarial prompt resistance.

The English summary is being prepared. Showing the original description.

Modelo de moderação especializado 7-8B do Meta para filtragem de entrada/saída de LLM. 6 categorias de segurança - violência/ódio, conteúdo sexual, armas, substâncias, automutilação, planejamento criminal. Precisão de 94-95%. Deploy com vLLM, HuggingFace, Sagemaker. Integra com NeMo Guardrails.

The English summary is being prepared. Showing the original description.

Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

The English summary is being prepared. Showing the original description.

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with unit tests per `write-code-eval`).

The English summary is being prepared. Showing the original description.