215 skills · sorted by relevance
Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.
Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.
Interactively gather parameters for `python -m cli.inference.throughput_optimizer`, generate a deployment simulation command, explain assumptions, ask for execution confirmation, then run the simulation and summarize the best parallel strategy. Use when the user wants to evaluate a model on one or more hardware profiles, compare hardware targets, compare aggregation versus disaggregation versus PD ratio optimization, choose TP/EP/MOE-DP search ranges, or obtain recommended concurrency, throughput, TTFT, and TPOT results from throughput modeling.
The English summary is being prepared. Showing the original description.
Run concurrent Omnigent conversations with real hosts and runners while replacing the LLM with a zero-latency mock. Read the generated report to explain response-time distributions, throughput, and failures, helping assess Omnigent's own processing overhead.
Analyze and optimize system throughput including request handling, data processing, and resource utilization. Use when identifying capacity limits or evaluating scaling strategies. Trigger with phrases like "analyze throughput", "optimize capacity", or "identify bottlenecks".
The English summary is being prepared. Showing the original description.
Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.
The English summary is being prepared. Showing the original description.
Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.
The English summary is being prepared. Showing the original description.
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
The English summary is being prepared. Showing the original description.
Turn speedup requests into a measured optimization process with limits on time, cost, and variants. Compare each change against a baseline and correctness checks, then retain the fastest safe option with reproducible commands and measurements.
Set up vLLM to serve large language models through an OpenAI-compatible API or process prompts in batches. The skill covers deployment, performance monitoring, and quantization, which reduces model memory requirements.
Build Python code for Azure Cosmos DB's NoSQL API to create, read, update, delete, and query documents. Covers authentication, containers for storing data, partition keys for distributing records, throughput settings, asynchronous operations, and error handling.
Check ten recurring problems before or during machine learning runs on NVIDIA DGX Spark. Diagnose startup failures, out-of-memory errors, and falling throughput, with guidance on compatible environments, resource use, numeric formats, and training across two devices.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Investigate slow TPU training on GKE by comparing diagnostic events with one-minute monitoring metrics. Separate memory or CPU bottlenecks from hardware and network issues, and identify the appropriate next steps.
Investigate Google Cloud network problems using traffic logs, performance metrics, and connection path tests. Identify blocked traffic, packet loss, and latency, summarize traffic volumes, or estimate VPC Flow Logs costs.
Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.
Run `doca_bench` (DOCA 2.7.0 or newer) to measure throughput, bulk latency, precision latency, or maximum bandwidth for RDMA, Compress, AES-GCM, SHA, DMA, EC, Ethernet, Comch, or GPUNetIO on a host or BlueField Arm. Use it to discover enabled benchmark libraries, capture a reproducible command/version/device/environment baseline, compare stable runs against a declared tolerance, or diagnose configuration, device-binding, workload-precondition, and measurement failures. Trigger for requests such as measuring BlueField compression speed, NIC RDMA throughput, crypto latency, or a pre-upgrade baseline. Do not use for application end-to-end timing, custom benchmark code, DOCA installation, or binary patches.
The English summary is being prepared. Showing the original description.
Use this skill when the user is building, running, or interpreting the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA kernel on the client posts RDMA WRITE work requests through the doca-gpunetio device-side surface to measure sustained GPU-driven WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user does not explicitly mention "doca-gpunetio-ib-write-bw" or "GPUNetIO" — typical implicit phrasings include "measure WRITE BW when the GPU posts the WRs", "BW swings between runs on the same flags", "is the NIC saturated or am I CPU-bound on the CUDA kernel", "meson compile fails for the GPUNetIO bw tool", "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated WRITE throughput vs CPU-initiated perftest". Refuse and route elsewhere for general doca-gpunetio library work, DOCA install, the GPU-initiated WRITE latency analog, the CPU-initiated upstream perftest, or application-level end-to-end throughput — those belong to other skills.
The English summary is being prepared. Showing the original description.
[omh] AI agent progress for managers: help managers inspect AI-agent progress, blockers, quality gates, and throughput levers. Use when the user says: agent-ops-review, agent ops review, agent productivity, operator productivity, manager view, quality dashboard, throughput review, agent work quality.
The English summary is being prepared. Showing the original description.
Design account-aware Alchemy throttling, bounded retries, and admission control from observed throughput rather than stale plan constants. Use when preventing or handling rate limits. Trigger with "Alchemy rate limit", "Alchemy throughput", or "fix Alchemy 429s".
The English summary is being prepared. Showing the original description.
Diagnose performance issues on Amazon SageMaker HyperPod clusters — uneven NCCL bandwidth across nodes and poor filesystem throughput. Read-only. Surfaces host-side signals (Xid, ECC, NVLink, EFA reachability, FSx saturation) and routes to the appropriate sibling skill (hyperpod-node-debugger, hyperpod-nccl, hyperpod-version-checker, hyperpod-issue-report) for any remediation. Triggers on uneven NCCL across nodes, straggler node, FSx slow, checkpoint slow, dataloader slow, filesystem bottleneck, FSx throughput, cross-AZ latency, topology mismatch.
The English summary is being prepared. Showing the original description.
Use when evaluating a newly adapted model's accuracy/eval score. Runs evalscope eval (or the test/srt/run_eval.py fallback) against an OpenAI-compatible endpoint on datasets like gsm8k / aime, producing per-dataset reports and a score. Accuracy only — throughput/latency benchmarks belong to model-speed-benchmark. Triggers on accuracy, eval, evalscope, gsm8k, aime, run_eval — not throughput.
The English summary is being prepared. Showing the original description.