“cuda”的搜索结果

Guides the use of RWKV, a language model architecture that carries information forward in a compact state. Covers streaming text generation, long document processing, fine-tuning, and memory comparisons with Transformers.

中文简介正在准备中。当前显示英语简介。

• Build streaming text generation
• Process long documents in chunks
• Compare memory needs with Transformers

Build and evaluate Mamba models for long sequences using state-space models, which summarize past inputs in an internal state. Covers setup, pretrained text generation, differences between Mamba-1 and Mamba-2, and speed comparisons with Transformers.

中文简介正在准备中。当前显示英语简介。

• Generate text with pretrained Mamba models
• Compare Mamba-1 and Mamba-2 configurations
• Benchmark generation speed against Transformers

Compress large language models to 4-bit weights to reduce GPU memory use and accelerate inference. Covers loading compressed models, configuring quantization, and using Transformers or vLLM, with guidance on hardware requirements and backend choices.

中文简介正在准备中。当前显示英语简介。

• Running large models with limited GPU memory
• Compressing a model with domain-specific sample text
• Serving AWQ models with vLLM

Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill explains model selection, batch processing, and memory-saving settings, with Python examples for connecting image understanding to language models.

中文简介正在准备中。当前显示英语简介。

• Generating captions for multiple images
• Building a question-answering system for images
• Searching an image collection with text

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

中文简介正在准备中。当前显示英语简介。

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Use SAELens to train and analyze sparse autoencoders, which separate neural network activity into features that may represent recognizable concepts. Explore pretrained features, evaluate reconstruction quality, and investigate how individual features influence language model outputs.

中文简介正在准备中。当前显示英语简介。

• Exploring features with a pretrained SAE
• Training and evaluating a custom SAE
• Studying overlapping concepts in model activity

Integrate Flash Attention into transformer models to reduce processing time and GPU memory use for long sequences. Covers PyTorch integration, advanced attention options, and benchmarks that compare speed, memory use, and output accuracy.

中文简介正在准备中。当前显示英语简介。

• Speeding up transformer training on long sequences
• Reducing GPU memory use during attention
• Adding Flash Attention to a PyTorch model

Helps speed up LLM text generation by comparing methods that predict and verify multiple tokens at once. Covers draft model selection, Medusa prediction heads, Lookahead Decoding settings, and deployment examples for inference services.

中文简介正在准备中。当前显示英语简介。

• Reducing chatbot response latency
• Comparing methods for faster code generation
• Selecting a small draft model for a larger LLM

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

中文简介正在准备中。当前显示英语简介。

• Clean web data for model training
• Remove exact and near duplicates from large text collections
• Redact personal information from training text

Check that changes to existing PEFT methods still work with saved training checkpoints. The skill guides compatibility tests, configuration checks, benchmark comparisons, and reporting, including comparisons of model outputs before and after a change.

中文简介正在准备中。当前显示英语简介。

• Checking compatibility after a forward method rewrite
• Changing state_dict structure while supporting old checkpoints
• Reviewing defaults for new configuration fields

Add new operators to MNN through a staged workflow covering data definitions, output shapes, computation, and tests. It prioritizes combining existing operators, with CPU implementations and additional hardware backends used when needed.

中文简介正在准备中。当前显示英语简介。

• Adding a new operator to MNN
• Implementing output shape calculations
• Combining existing operators for new operations

Triage incorrect MNN outputs and regressions by matching symptoms to focused debugging guides. Use backend comparisons, minimal reproductions, and controlled experiments to investigate memory issues, quantization, concurrency, numeric limits, caches, and inconsistent results across runs.

中文简介正在准备中。当前显示英语简介。

• Investigating garbled or degraded MNN output
• Comparing results across execution backends
• Tracking regressions after optimization changes

Run TVM's TIRx compiler tests alongside tirx-kernels import and correctness checks. It verifies the local build, selects GPUs, runs parallel tests, saves full logs, and guides failure investigation and result reporting.

中文简介正在准备中。当前显示英语简介。

• Testing TIRx compiler and kernel changes
• Checking tirx-kernels imports and registry entries
• Running parallel tests across selected GPUs

Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill guides model selection, batch processing, and memory optimization, with examples for understanding images without training for each specific task.

中文简介正在准备中。当前显示英语简介。

• Generate captions for multiple images
• Build a question-answering system for images
• Search an image collection using text

Build image generation workflows in Python with Stable Diffusion and HuggingFace Diffusers. The guide covers text prompts, transforming existing images, replacing selected regions, controlling composition, applying learned styles, and adjusting memory use during generation.

中文简介正在准备中。当前显示英语简介。

• Generating images from text descriptions
• Changing the style of an existing image
• Replacing a selected region of an image

Reduce LLM generation delays with speculative decoding, which proposes tokens ahead of verification. This skill compares small draft models, Medusa, and Lookahead Decoding, with implementation examples, tuning guidance, and deployment using vLLM.

中文简介正在准备中。当前显示英语简介。

• Reducing response delays in chatbots
• Speeding up code generation
• Choosing a small model to draft candidate tokens

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

中文简介正在准备中。当前显示英语简介。

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Guide knowledge distillation, where a smaller language model learns from a larger teacher. Covers training from teacher responses or prediction scores, adjusting training settings, and comparing outputs to support model compression and specialized training.

中文简介正在准备中。当前显示英语简介。

• Compressing a large language model into a smaller model
• Training a specialized model from teacher-generated examples
• Comparing distillation methods and training settings

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

中文简介正在准备中。当前显示英语简介。

• Clean web data for model training
• Remove exact and near duplicates from large text collections
• Redact personal information from training text

Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.

中文简介正在准备中。当前显示英语简介。

• Training a model that exceeds one GPU's memory
• Adding FSDP2 to an existing training script
• Configuring mixed precision and CPU offloading

Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.

中文简介正在准备中。当前显示英语简介。

• Load a larger model within available GPU memory
• Compare 4-bit and 8-bit quantization settings
• Fine-tune a quantized model with QLoRA

Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.

中文简介正在准备中。当前显示英语简介。

• Running GPU inference without managing servers
• Deploying a model as an automatically scaling API
• Processing large batches in parallel

Guides you through setting up RWKV, generating text, processing long documents in chunks, and fine-tuning models. Explains how to carry context between steps and compare memory use with Transformer models.

中文简介正在准备中。当前显示英语简介。

• Build streaming text generation
• Process long documents in chunks
• Plan RWKV fine-tuning

Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.

中文简介正在准备中。当前显示英语简介。

• Choose model architectures for the available data type and scale
• Configure optimizers and learning-rate schedules
• Investigate loss spikes, NaNs, and GPU memory shortages