'cuda' 검색 결과

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Guides the use of RWKV, a language model architecture that carries information forward in a compact state. Covers streaming text generation, long document processing, fine-tuning, and memory comparisons with Transformers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build streaming text generation
• Process long documents in chunks
• Compare memory needs with Transformers

Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill explains model selection, batch processing, and memory-saving settings, with Python examples for connecting image understanding to language models.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Generating captions for multiple images
• Building a question-answering system for images
• Searching an image collection with text

Helps speed up LLM text generation by comparing methods that predict and verify multiple tokens at once. Covers draft model selection, Medusa prediction heads, Lookahead Decoding settings, and deployment examples for inference services.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing chatbot response latency
• Comparing methods for faster code generation
• Selecting a small draft model for a larger LLM

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Clean web data for model training
• Remove exact and near duplicates from large text collections
• Redact personal information from training text

Check that changes to existing PEFT methods still work with saved training checkpoints. The skill guides compatibility tests, configuration checks, benchmark comparisons, and reporting, including comparisons of model outputs before and after a change.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Checking compatibility after a forward method rewrite
• Changing state_dict structure while supporting old checkpoints
• Reviewing defaults for new configuration fields

Add new operators to MNN through a staged workflow covering data definitions, output shapes, computation, and tests. It prioritizes combining existing operators, with CPU implementations and additional hardware backends used when needed.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Adding a new operator to MNN
• Implementing output shape calculations
• Combining existing operators for new operations

Triage incorrect MNN outputs and regressions by matching symptoms to focused debugging guides. Use backend comparisons, minimal reproductions, and controlled experiments to investigate memory issues, quantization, concurrency, numeric limits, caches, and inconsistent results across runs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating garbled or degraded MNN output
• Comparing results across execution backends
• Tracking regressions after optimization changes

Run TVM's TIRx compiler tests alongside tirx-kernels import and correctness checks. It verifies the local build, selects GPUs, runs parallel tests, saves full logs, and guides failure investigation and result reporting.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Testing TIRx compiler and kernel changes
• Checking tirx-kernels imports and registry entries
• Running parallel tests across selected GPUs

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Reduce LLM generation delays with speculative decoding, which proposes tokens ahead of verification. This skill compares small draft models, Medusa, and Lookahead Decoding, with implementation examples, tuning guidance, and deployment using vLLM.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing response delays in chatbots
• Speeding up code generation
• Choosing a small model to draft candidate tokens

Guide knowledge distillation, where a smaller language model learns from a larger teacher. Covers training from teacher responses or prediction scores, adjusting training settings, and comparing outputs to support model compression and specialized training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compressing a large language model into a smaller model
• Training a specialized model from teacher-generated examples
• Comparing distillation methods and training settings

Build image captioning, visual question answering, and text-based image search with BLIP-2. The skill guides model selection, batch processing, and memory optimization, with examples for understanding images without training for each specific task.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Generate captions for multiple images
• Build a question-answering system for images
• Search an image collection using text

Build image generation workflows in Python with Stable Diffusion and HuggingFace Diffusers. The guide covers text prompts, transforming existing images, replacing selected regions, controlling composition, applying learned styles, and adjusting memory use during generation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Generating images from text descriptions
• Changing the style of an existing image
• Replacing a selected region of an image

Add PyTorch FSDP2 to training scripts so model state can be split across GPUs. Covers distributed initialization, mixed precision, CPU offloading, and checkpoint saving and loading for models that exceed a single GPU's memory.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Training a model that exceeds one GPU's memory
• Adding FSDP2 to an existing training script
• Configuring mixed precision and CPU offloading

Apply HQQ quantization, which stores language model weights using fewer bits without calibration examples. Covers precision settings, inference backends, saving compressed models, deployment with HuggingFace Transformers or vLLM, and further training with LoRA.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compressing a model without calibration data
• Comparing 4-bit and 2-bit quantization settings
• Running quantized models with HuggingFace Transformers or vLLM

Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a local LLM on an Apple Silicon Mac
• Trying LLM inference on a CPU-only machine
• Choosing a model format that fits available memory

Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose model architectures for the available data type and scale
• Configure optimizers and learning-rate schedules
• Investigate loss spikes, NaNs, and GPU memory shortages

Guide language model training with TRL, from learning to follow instructions to matching human preferences. Covers method selection, training data, reward models that score responses, evaluation, and adjustments for memory limits or unstable training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train a model on instruction and response examples
• Align responses using preferred and rejected answer pairs
• Build a training pipeline using human feedback

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Clean web data for model training
• Remove exact and near duplicates from large text collections
• Redact personal information from training text

Use SAELens to train and analyze sparse autoencoders, which break model activations into a small set of interpretable features. Explore pretrained features, evaluate reconstruction quality, troubleshoot training, and test how individual features influence generated text.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Exploring features in a pretrained SAE
• Training an SAE for a selected model layer
• Investigating inactive features and reconstruction errors

Compress large language model weights to four bits with AWQ, a method that uses model activation patterns to limit quality loss. Follow workflows for quantization, loading compressed models, and serving them with vLLM using less GPU memory.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running large language models with limited GPU memory
• Quantizing a model with domain-specific text
• Serving an AWQ model with vLLM

Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing GPU memory use with long sequences
• Integrating Flash Attention into PyTorch models
• Comparing runtime and outputs before and after changes

Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Load a larger model within available GPU memory
• Compare 4-bit and 8-bit quantization settings
• Fine-tune a quantized model with QLoRA