“gpu”的搜索结果

Compress large language model weights to four bits with AWQ, a method that uses model activation patterns to limit quality loss. Follow workflows for quantization, loading compressed models, and serving them with vLLM using less GPU memory.

中文简介正在准备中。当前显示英语简介。

• Running large language models with limited GPU memory
• Quantizing a model with domain-specific text
• Serving an AWQ model with vLLM

Use this skill for hands-on DOCA GPI programming — wiring a GPU-Packet-Initiator context so a CUDA kernel drives RDMA queues directly from GPU memory without host CPU mediation. Covers picking GPI vs doca-gpunetio, the doca_gpi / domain / channel object model, the GPU-side handle handoff (doca_gpu_gpi_channel*), attaching GPU memory to a GPI domain, the domain and channel attribute objects, and debugging DOCA_ERROR_* from doca_gpi_* calls. Trigger even when the user does not explicitly mention "DOCA GPI" — implicit phrasings include "my CUDA kernel needs to post RDMA directly from GPU memory", "DOCA_ERROR_* from doca_gpi_gpu_channel_get", "how do I hand a GPU handle to my CUDA kernel", "how many channels can a GPI domain hold", or "GPU kernel driving RDMA without the host CPU on the path". Refuse and route elsewhere for the doca-gpunetio Send/Receive surface, the doca-rdma queue lifecycle, DPA-side initiation (doca-rdmi), or the CUDA programming model — those belong to other skills.

中文简介正在准备中。当前显示原文说明。

AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process isolation, `gr.State` semantics across the worker boundary, no `torch.compile` (use AoTI instead), CUDA wheel-only builds (no `nvcc` at build or runtime), large vs xlarge sizing, and dynamic duration callables. Make sure to use this skill whenever the user mentions ZeroGPU, `@spaces.GPU`, or the `spaces` Python package, or hits ZeroGPU-specific code errors like `PicklingError` across the worker boundary, `illegal duration`, or `flash-attn` wheel-build failures — even when the user does not explicitly ask for ZeroGPU coding guidance. Trigger on `import spaces` or `@spaces.GPU` in code.

中文简介正在准备中。当前显示原文说明。

Investigate recurring SGLang automated test failures by comparing logs, commit history, and GPU environments. Narrow down code changes or machine-specific problems, optionally reproduce failures remotely, and summarize the evidence in a structured report.

中文简介正在准备中。当前显示英语简介。

• Finding the change behind a recurring test failure
• Checking whether failures depend on a GPU or runner
• Reproducing a test failure on a remote GPU server

Build similarity search with FAISS to find nearby matches in large collections of numeric vectors. The skill guides index selection, memory reduction, GPU use, and saving indexes while balancing search speed and accuracy.

中文简介正在准备中。当前显示英语简介。

• Find similar items in large vector datasets
• Choose an index for search speed and accuracy
• Reduce memory use for vector search

Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.

中文简介正在准备中。当前显示英语简介。

• Choosing a method for training models on human preferences
• Setting up PPO training with a reward model
• Configuring GRPO training without a critic model

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Adapt PyTorch training scripts for multiple GPUs or machines with Accelerate. This skill guides code changes, configuration, and launch commands, including mixed precision, which combines numerical formats, and DeepSpeed or FSDP integration for distributed training.

中文简介正在准备中。当前显示英语简介。

• Moving single-GPU training to multiple GPUs
• Configuring mixed precision training
• Setting up DeepSpeed or FSDP

Investigate interruptions affecting GPU and TPU workloads on GKE using maintenance labels, monitoring queries, logs, and node scheduling restrictions. The skill helps determine whether host maintenance caused a disruption and recommends workload protection measures when maintenance is confirmed or expected.

中文简介正在准备中。当前显示英语简介。

• Investigating unexpected GPU or TPU workload interruptions
• Checking upcoming host maintenance on GKE nodes
• Tracking maintenance interruptions with PromQL

Improve MNN's Vulkan backend by separating CPU scheduling bottlenecks from GPU computation costs. It guides operator and shader changes, new feature integration with compatibility fallbacks, and device testing to verify both correctness and end-to-end performance.

中文简介正在准备中。当前显示英语简介。

• Finding CPU or GPU bottlenecks in Vulkan inference
• Optimizing a specific operator or shader
• Integrating cooperative matrix or subgroup features

Build similarity search with FAISS to find nearby matches in large collections of numeric vectors. The skill guides index selection, memory reduction, GPU use, and saving indexes while balancing search speed and accuracy.

中文简介正在准备中。当前显示英语简介。

• Find similar items in large vector datasets
• Choose an index for search speed and accuracy
• Reduce memory use for vector search

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

中文简介正在准备中。当前显示英语简介。

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.

中文简介正在准备中。当前显示英语简介。

• Running a local LLM on an Apple Silicon Mac
• Trying LLM inference on a CPU-only machine
• Choosing a model format that fits available memory

Adapt existing PyTorch training scripts to run across multiple GPUs or machines with Accelerate. This skill covers device setup, distributed training configuration, and mixed precision, which uses lower-precision numbers to reduce memory use.

中文简介正在准备中。当前显示英语简介。

• Adapt a single-GPU training script for multiple GPUs
• Configure mixed precision to reduce training memory use
• Set up training with DeepSpeed or FSDP

Build and maintain machine learning apps on Hugging Face Spaces. Choose a suitable app framework and computing hardware, load models, deploy files, and check logs and outputs to diagnose build or runtime problems.

中文简介正在准备中。当前显示英语简介。

• Publishing a machine learning demo
• Adapting an existing Gradio app for ZeroGPU
• Diagnosing Space build or runtime failures

Plan how to split AI models and workloads across multiple GPUs. Compare memory needs, computation, communication, and waiting time to estimate completion times and choose a layout suited to response time, throughput, or cost goals.

中文简介正在准备中。当前显示英语简介。

• Running models that exceed a single GPU's memory
• Comparing ways to split work across GPUs
• Estimating communication costs within and between servers

协助配置vLLM，将大语言模型作为API提供服务或批量生成文本，涵盖部署、性能监控、量化和多GPU分工，也用于排查显存不足与响应缓慢。

• 提供兼容OpenAI的API
• 批量处理大量提示词
• 在有限显存下运行模型

根据歌词及曲风、乐器标签在本地生成歌曲，指导安装、模型下载、输入整理和节省显存的设置，并协助将生成结果保存为MP3文件。

• 将自写歌词生成歌曲
• 通过标签尝试曲风和乐器
• 在本地生成音乐

Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.

中文简介正在准备中。当前显示英语简介。

• Identify GPU operations that dominate inference time
• Compare input processing and token generation
• Trace GPU operations back to Python code

Investigate SGLang runs that freeze across multiple GPUs. It uses stack traces, communication logs, GPU memory dumps, and comparisons between processes to locate state differences, trace their causes, and guide fixes.

中文简介正在准备中。当前显示英语简介。

• Locate where a multi-GPU inference run freezes
• Compare logs from distributed processes
• Trace the first difference in process state

Configure machine learning training and batch jobs across multiple clouds with SkyPilot. The skill guides GPU selection, lower-cost cloud and region choices, and automatic recovery when discounted spot instances are interrupted.

中文简介正在准备中。当前显示英语简介。

• Choose a lower-cost cloud for GPU training
• Run long training jobs with recovery after interruptions
• Configure distributed training across multiple machines

Organize PyTorch training code with LightningModule and Trainer. Follow examples for validation, logging, model saving, and distributed training across multiple GPUs, with callbacks that add monitoring or stop training when progress stalls.

中文简介正在准备中。当前显示英语简介。

• Converting an existing PyTorch training loop
• Adding validation, logs, and model checkpoints
• Configuring training across multiple GPUs

Run batches of machine learning experiments on remote GPUs over SSH, varying settings and random seeds. The skill schedules jobs, tracks progress, retries GPU memory failures, and cleans up leftover sessions before starting subsequent batches.

中文简介正在准备中。当前显示英语简介。

• Run experiments across multiple random seeds
• Test combinations of model settings
• Chain teacher training and student distillation

Configure machine learning training and batch jobs across multiple clouds with SkyPilot. The skill guides GPU selection, lower-cost cloud and region choices, and automatic recovery when discounted spot instances are interrupted.

中文简介正在准备中。当前显示英语简介。

• Choose a lower-cost cloud for GPU training
• Run long training jobs with recovery after interruptions
• Configure distributed training across multiple machines