Results for “gpu”

Write and review Gradio demos for Hugging Face ZeroGPU. Configure GPU time limits and dependencies, and troubleshoot errors caused by separate worker processes, shared state, concurrent requests, or CUDA package installation.

• Review GPU handlers in a Gradio demo
• Tune GPU duration and quota usage
• Fix PicklingError and stale state across worker processes

Optimize scientific Python workloads for NVIDIA GPUs by choosing suitable libraries and reducing bottlenecks. Compare outputs with the CPU version and measure complete processing time, including data transfers, to determine whether the GPU implementation provides a useful improvement.

• Assess whether slow Python processing would benefit from a GPU
• Move NumPy or pandas workloads to suitable GPU libraries
• Validate GPU results against CPU outputs

Evaluate a machine learning GPU cluster during a cloud trial or acceptance test. Measure GPU computing performance, communication within and between machines, and local and shared storage, then compile the results into a dated Markdown report.

• Evaluating trial GPU machines from a cloud provider
• Running acceptance tests for a GPU cluster
• Measuring GPU performance under load from neighboring GPUs

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Guides setup and operation of Lambda Labs GPU instances for machine learning training and inference. Covers SSH access, persistent storage, API automation, and multi-GPU or multi-node training, including how to retain checkpoints and terminate idle instances.

• Setting up a GPU instance for LLM fine-tuning
• Keeping datasets and checkpoints between sessions
• Automating instance launch and termination through the API

Build desktop applications with GPUI Kit, a Rust framework. Guides setup, component selection, state management, themes, and background work, and explains UI integration tests that simulate input and check component behavior.

• Setting up a GPUI Kit desktop app
• Implementing forms and data tables
• Organizing state, events, and background tasks

Run model training, fine-tuning, inference, and batch jobs on Modal’s cloud GPUs. Estimate hardware needs and cost, generate Python launch scripts, run or deploy workloads, monitor logs, and retrieve results without managing remote servers.

• Test CUDA code without a local GPU
• Run model training or LoRA fine-tuning on cloud GPUs
• Deploy a model inference API or vLLM service

Guides setup and operation of Lambda Labs GPU instances for machine learning training and inference. Covers SSH access, persistent storage, API automation, and multi-GPU or multi-node training, including how to retain checkpoints and terminate idle instances.

• Setting up a GPU instance for LLM fine-tuning
• Keeping datasets and checkpoints between sessions
• Automating instance launch and termination through the API

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Choose SGLang Diffusion settings for a specific model, GPU, and memory budget. Compare generation speed, memory use, and output quality to select command-line options, including CPU offloading, multiple GPUs, caching, and reduced precision.

• Choosing generation settings for a specific model and GPU
• Reducing GPU memory use to fit a memory budget
• Comparing performance settings across multiple GPUs

Rent and manage vast.ai GPUs based on the training task rather than a chosen hardware model. The skill estimates requirements, compares offers by total expected cost, prepares the selected instance, and retrieves results before destroying it.

• Choosing a cloud GPU for a training task
• Comparing rental offers by estimated total cost
• Preparing a rented GPU environment for experiments

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Detect and validate GPU availability for MATLAB GPU computing. Use when the user can't use their GPU, or is setting up or selecting a GPU. Triggers on: gpuDevice, GPU setup, check GPU, GPU not found, GPU not working, can't use GPU, GPU not available, unable to find a supported GPU device, compatible GPU, canUseGPU, validateGPU.

The English summary is being prepared. Showing the original description.

Set up GPU servers on Lambda Labs for machine learning training and inference. Follow workflows for SSH access, persistent storage, Python API operations, and distributed training that spreads work across multiple GPUs or servers.

• Set up a GPU server for long training jobs
• Fine-tune a large language model across multiple GPUs
• Keep datasets and checkpoints between sessions

Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.

• Running GPU inference without managing servers
• Deploying a model as an automatically scaling API
• Processing large batches in parallel

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.

• Diagnosing slow training and GPU memory shortages
• Investigating loss spikes and invalid numerical values
• Debugging communication hangs across GPUs and machines

Launch machine learning experiments on local GPUs, remote servers, Vast.ai, or Modal. It checks the configured environment, deploys code, saves logs, and verifies startup, with optional W&B metric logging and automatic Vast.ai result collection and instance cleanup.

• Run training jobs on a local GPU
• Deploy experiments to an SSH server
• Run experiments on Vast.ai and collect results

Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.

• Running GPU inference without managing servers
• Deploying a model as an automatically scaling API
• Processing large batches in parallel

Set up and optimize large language model inference, or response generation, on NVIDIA GPUs. Use quantization, batch processing, and multiple GPUs to improve serving performance, and expose models through a chat completion API.

• Serving a language model through a chat completion API
• Processing many prompts in batches
• Configuring quantization to reduce memory use

Operate NVIDIA Nsight Graphics from the command line to capture application frames and GPU activity. Summarize performance traces, analyze existing capture files through ngfx-replay, and check installed versions and compatibility before starting an investigation.

• Check installed Nsight Graphics versions and compatibility
• Capture frames from a graphics application
• Summarize GPU activity and investigate performance bottlenecks

Measure scripts, running processes, or whole systems to find performance bottlenecks. Select suitable tools, add measurement code when needed, and report CPU, GPU, memory, and communication findings with prioritized recommendations and a record of file changes.

• Finding slow functions in a Python script
• Investigating CPU and GPU memory usage
• Measuring communication overhead across multiple GPUs

Guide large language model training with OpenRLHF using human preference feedback. Covers training method selection, distributed GPU setup, reward model training, and memory tuning, with examples for PPO, GRPO, and DPO.

• Choosing a training method for human preference alignment
• Setting up distributed training across multiple GPUs
• Training a reward model before PPO