204 skills · sorted by relevance
Guides the use of RWKV, a language model architecture that carries information forward in a compact state. Covers streaming text generation, long document processing, fine-tuning, and memory comparisons with Transformers.
Build and evaluate Mamba models for long sequences using state-space models, which summarize past inputs in an internal state. Covers setup, pretrained text generation, differences between Mamba-1 and Mamba-2, and speed comparisons with Transformers.
Compress large language models to 4-bit weights to reduce GPU memory use and accelerate inference. Covers loading compressed models, configuring quantization, and using Transformers or vLLM, with guidance on hardware requirements and backend choices.
Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill explains model selection, batch processing, and memory-saving settings, with Python examples for connecting image understanding to language models.
Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.
Use SAELens to train and analyze sparse autoencoders, which separate neural network activity into features that may represent recognizable concepts. Explore pretrained features, evaluate reconstruction quality, and investigate how individual features influence language model outputs.
Integrate Flash Attention into transformer models to reduce processing time and GPU memory use for long sequences. Covers PyTorch integration, advanced attention options, and benchmarks that compare speed, memory use, and output accuracy.
Helps speed up LLM text generation by comparing methods that predict and verify multiple tokens at once. Covers draft model selection, Medusa prediction heads, Lookahead Decoding settings, and deployment examples for inference services.
Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.
Check that changes to existing PEFT methods still work with saved training checkpoints. The skill guides compatibility tests, configuration checks, benchmark comparisons, and reporting, including comparisons of model outputs before and after a change.
Add new operators to MNN through a staged workflow covering data definitions, output shapes, computation, and tests. It prioritizes combining existing operators, with CPU implementations and additional hardware backends used when needed.
Triage incorrect MNN outputs and regressions by matching symptoms to focused debugging guides. Use backend comparisons, minimal reproductions, and controlled experiments to investigate memory issues, quantization, concurrency, numeric limits, caches, and inconsistent results across runs.
Run TVM's TIRx compiler tests alongside tirx-kernels import and correctness checks. It verifies the local build, selects GPUs, runs parallel tests, saves full logs, and guides failure investigation and result reporting.
Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill guides model selection, batch processing, and memory optimization, with examples for understanding images without training for each specific task.
Build image generation workflows in Python with Stable Diffusion and HuggingFace Diffusers. The guide covers text prompts, transforming existing images, replacing selected regions, controlling composition, applying learned styles, and adjusting memory use during generation.
Reduce LLM generation delays with speculative decoding, which proposes tokens ahead of verification. This skill compares small draft models, Medusa, and Lookahead Decoding, with implementation examples, tuning guidance, and deployment using vLLM.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Guide knowledge distillation, where a smaller language model learns from a larger teacher. Covers training from teacher responses or prediction scores, adjusting training settings, and comparing outputs to support model compression and specialized training.
Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.
Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.
Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.
Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.
Guides you through setting up RWKV, generating text, processing long documents in chunks, and fine-tuning models. Explains how to carry context between steps and compare memory use with Transformer models.
Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.