Orchestra Research

Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.

中文简介正在准备中。当前显示英语简介。

• Running GPU inference without managing servers
• Deploying a model as an automatically scaling API
• Processing large batches in parallel

Build text tokenizers, which split text into units for AI models, directly from raw multilingual text. Learn custom vocabularies with BPE or Unigram, convert text to pieces or numeric IDs, and decode the results.

中文简介正在准备中。当前显示英语简介。

• Preparing text for multilingual AI models
• Training a vocabulary from Japanese text
• Reusing a tokenizer for reproducible text splitting

Fine-tune language models with TRL using GRPO, a method that compares scored responses to the same prompt. Covers training data, scoring functions, GPU-conscious settings, and monitoring for tasks with measurable correctness or required output formats.

中文简介正在准备中。当前显示英语简介。

• Train models to follow JSON or XML output formats
• Reward correct answers in math or coding tasks
• Balance correctness, format, and style during training

Prepare large text collections for AI models with fast tokenizers, which split text into model inputs. Train custom vocabularies, trace tokens to original text positions, adjust input lengths, and integrate the results with transformers.

中文简介正在准备中。当前显示英语简介。

• Tokenizing large text collections
• Training a custom vocabulary
• Mapping extracted names or answers back to original text

Get guidance on adapting LLMs through additional training with LLaMA-Factory. The skill covers a visual interface requiring no code, LoRA and QLoRA training methods, and documentation-based help with implementation, features, and debugging.

中文简介正在准备中。当前显示英语简介。

• Learning additional training through a visual interface
• Exploring LoRA and QLoRA training methods
• Checking supported models and multimodal features

Guides experiments that change internal values in PyTorch models to test what causes their behavior. Uses pyvene to swap or remove activations, locate influential components, train interventions, and save experiments for reuse.

中文简介正在准备中。当前显示英语简介。

• Locating model components that affect an answer
• Comparing outputs after swapping internal activations
• Training interventions to investigate causal structure

Investigate how transformer language models produce predictions with TransformerLens. This skill guides inspection of intermediate values and attention patterns, plus experiments that replace internal values to test their effects and identify components involved in model behavior.

中文简介正在准备中。当前显示英语简介。

• Inspecting attention patterns and intermediate values
• Testing output changes by replacing internal activations
• Analyzing attention head contributions to predictions

Coordinate AI research from literature review and hypothesis testing to findings and paper drafts. The skill alternates focused experiments with broader reflection, maintains project records, and directs specialized skills to carry out research tasks.

中文简介正在准备中。当前显示英语简介。

• Turning a research question into testable hypotheses
• Managing repeated experiments across multiple hypotheses
• Resuming research from recorded findings and project state

Implement Mamba, a model architecture that processes sequences through an internal state with computation scaling linearly with input length. Covers pretrained text generation, Mamba-1 and Mamba-2 configuration, speed comparisons with Transformers, and GPU memory troubleshooting.

中文简介正在准备中。当前显示英语简介。

• Explore models for long sequences or streaming applications
• Generate text with pretrained Mamba models
• Compare Mamba-1 and Mamba-2 configurations

Learn how GPT language models work by preparing text, training a small model, and generating samples. Covers CPU experiments, custom datasets, further training of pretrained GPT-2 models, and reproducing GPT-2 with multiple GPUs.

中文简介正在准备中。当前显示英语简介。

• Train a small model on Shakespeare using a CPU
• Prepare custom text for model training
• Further train a pretrained GPT-2 model

Use SAELens to train and analyze sparse autoencoders, which break model activations into a small set of interpretable features. Explore pretrained features, evaluate reconstruction quality, troubleshoot training, and test how individual features influence generated text.

中文简介正在准备中。当前显示英语简介。

• Exploring features in a pretrained SAE
• Training an SAE for a selected model layer
• Investigating inactive features and reconstruction errors

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

中文简介正在准备中。当前显示英语简介。

• Clean web data for model training
• Remove exact and near duplicates from large text collections
• Redact personal information from training text

Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.

中文简介正在准备中。当前显示英语简介。

• Preprocessing datasets larger than memory
• Running batch predictions across multiple GPUs
• Preparing image and tabular data for training

Guides large language model reinforcement learning with miles, including MoE models that route work to specialized components. Covers low-precision training, matching training and inference behavior, and accelerating training response generation with draft models.

中文简介正在准备中。当前显示英语简介。

• Setting up reinforcement learning for large MoE models
• Configuring FP8 or INT4 training
• Checking consistency between training and inference

Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.

中文简介正在准备中。当前显示英语简介。

• Choosing a training method for human preference alignment
• Setting up distributed training across multiple GPUs
• Training a reward model before PPO

Guide language model training with SimPO using pairs of preferred and rejected answers. Covers configuration, learning rate tuning, and memory troubleshooting for preference alignment, which adjusts model responses toward human preferences without a separate reference model.

中文简介正在准备中。当前显示英语简介。

• Training on preferred and rejected answer pairs
• Exploring an alternative to DPO without a reference model
• Tuning preference training for math and code tasks

Set up reinforcement learning, training guided by rewards, for language models with slime. The skill covers Megatron-LM training, SGLang response generation, data preparation, asynchronous execution, and custom workflows for agents that use tools across multiple turns.

中文简介正在准备中。当前显示英语简介。

• Prepare GRPO training for GLM or Qwen3 models
• Overlap response generation and model training
• Train agents with multi-turn tool use

Guides reinforcement learning, where models learn from rewards, using torchforge and PyTorch. Covers reward design, custom loss functions, and distributed training so researchers can experiment with algorithms while keeping infrastructure concerns separate.

中文简介正在准备中。当前显示英语简介。

• Train mathematical reasoning models with GRPO
• Experiment with custom loss functions
• Scale training across multiple GPUs or nodes

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

中文简介正在准备中。当前显示英语简介。

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.

中文简介正在准备中。当前显示英语简介。

• Train math reasoning models with GRPO and answer-based rewards
• Configure PPO training with a separate value-estimation model
• Run large-model training across multiple GPU nodes

Build safety checks for LLM prompts and responses using Meta's LlamaGuard. This skill covers content classification, batch moderation, serving a moderation API, and connecting checks to NeMo Guardrails.

中文简介正在准备中。当前显示英语简介。

• Screening user prompts before sending them to an LLM
• Checking generated responses before displaying them
• Building an API for content safety checks

Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.

中文简介正在准备中。当前显示英语简介。

• Adapt a single-GPU training script for multiple GPUs
• Configure mixed precision to reduce training memory use
• Set up training with DeepSpeed or FSDP

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

中文简介正在准备中。当前显示英语简介。

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

中文简介正在准备中。当前显示英语简介。

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput