221 个 · 按相关度排序
Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.
中文简介正在准备中。当前显示英语简介。
Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.
中文简介正在准备中。当前显示英语简介。
Interactively gather parameters for `python -m cli.inference.throughput_optimizer`, generate a deployment simulation command, explain assumptions, ask for execution confirmation, then run the simulation and summarize the best parallel strategy. Use when the user wants to evaluate a model on one or more hardware profiles, compare hardware targets, compare aggregation versus disaggregation versus PD ratio optimization, choose TP/EP/MOE-DP search ranges, or obtain recommended concurrency, throughput, TTFT, and TPOT results from throughput modeling.
中文简介正在准备中。当前显示原文说明。
Run concurrent Omnigent conversations with real hosts and runners while replacing the LLM with a zero-latency mock. Read the generated report to explain response-time distributions, throughput, and failures, helping assess Omnigent's own processing overhead.
中文简介正在准备中。当前显示英语简介。
Analyze and optimize system throughput including request handling, data processing, and resource utilization. Use when identifying capacity limits or evaluating scaling strategies. Trigger with phrases like "analyze throughput", "optimize capacity", or "identify bottlenecks".
中文简介正在准备中。当前显示原文说明。
Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.
中文简介正在准备中。当前显示原文说明。
Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.
中文简介正在准备中。当前显示原文说明。
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
中文简介正在准备中。当前显示原文说明。
将性能优化拆解为现状测量、单个假设的方案比较和正确性验证，在时间与费用上限内选出测得最快且安全的方案，并保留可重复执行的命令、配置和记录。
协助配置vLLM，将大语言模型作为API提供服务或批量生成文本，涵盖部署、性能监控、量化和多GPU分工，也用于排查显存不足与响应缓慢。
Build Python code for Azure Cosmos DB's NoSQL API to create, read, update, delete, and query documents. Covers authentication, containers for storing data, partition keys for distributing records, throughput settings, asynchronous operations, and error handling.
中文简介正在准备中。当前显示英语简介。
Check ten recurring problems before or during machine learning runs on NVIDIA DGX Spark. Diagnose startup failures, out-of-memory errors, and falling throughput, with guidance on compatible environments, resource use, numeric formats, and training across two devices.
中文简介正在准备中。当前显示英语简介。
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
中文简介正在准备中。当前显示英语简介。
Investigate slow TPU training on GKE by comparing diagnostic events with one-minute monitoring metrics. Separate memory or CPU bottlenecks from hardware and network issues, and identify the appropriate next steps.
中文简介正在准备中。当前显示英语简介。
Investigate Google Cloud network problems using traffic logs, performance metrics, and connection path tests. Identify blocked traffic, packet loss, and latency, summarize traffic volumes, or estimate VPC Flow Logs costs.
中文简介正在准备中。当前显示英语简介。
Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.
中文简介正在准备中。当前显示英语简介。
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
中文简介正在准备中。当前显示英语简介。
Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.
中文简介正在准备中。当前显示英语简介。
Plan how to split AI models and workloads across multiple GPUs. Compare memory needs, computation, communication, and waiting time to estimate completion times and choose a layout suited to response time, throughput, or cost goals.
中文简介正在准备中。当前显示英语简介。
Run `doca_bench` (DOCA 2.7.0 or newer) to measure throughput, bulk latency, precision latency, or maximum bandwidth for RDMA, Compress, AES-GCM, SHA, DMA, EC, Ethernet, Comch, or GPUNetIO on a host or BlueField Arm. Use it to discover enabled benchmark libraries, capture a reproducible command/version/device/environment baseline, compare stable runs against a declared tolerance, or diagnose configuration, device-binding, workload-precondition, and measurement failures. Trigger for requests such as measuring BlueField compression speed, NIC RDMA throughput, crypto latency, or a pre-upgrade baseline. Do not use for application end-to-end timing, custom benchmark code, DOCA installation, or binary patches.
中文简介正在准备中。当前显示原文说明。
Use this skill when the user is building, running, or interpreting the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA kernel on the client posts RDMA WRITE work requests through the doca-gpunetio device-side surface to measure sustained GPU-driven WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user does not explicitly mention "doca-gpunetio-ib-write-bw" or "GPUNetIO" — typical implicit phrasings include "measure WRITE BW when the GPU posts the WRs", "BW swings between runs on the same flags", "is the NIC saturated or am I CPU-bound on the CUDA kernel", "meson compile fails for the GPUNetIO bw tool", "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated WRITE throughput vs CPU-initiated perftest". Refuse and route elsewhere for general doca-gpunetio library work, DOCA install, the GPU-initiated WRITE latency analog, the CPU-initiated upstream perftest, or application-level end-to-end throughput — those belong to other skills.
中文简介正在准备中。当前显示原文说明。
[omh] AI agent progress for managers: help managers inspect AI-agent progress, blockers, quality gates, and throughput levers. Use when the user says: agent-ops-review, agent ops review, agent productivity, operator productivity, manager view, quality dashboard, throughput review, agent work quality.
中文简介正在准备中。当前显示原文说明。
Design account-aware Alchemy throttling, bounded retries, and admission control from observed throughput rather than stale plan constants. Use when preventing or handling rate limits. Trigger with "Alchemy rate limit", "Alchemy throughput", or "fix Alchemy 429s".
中文简介正在准备中。当前显示原文说明。
Diagnose performance issues on Amazon SageMaker HyperPod clusters — uneven NCCL bandwidth across nodes and poor filesystem throughput. Read-only. Surfaces host-side signals (Xid, ECC, NVLink, EFA reachability, FSx saturation) and routes to the appropriate sibling skill (hyperpod-node-debugger, hyperpod-nccl, hyperpod-version-checker, hyperpod-issue-report) for any remediation. Triggers on uneven NCCL across nodes, straggler node, FSx slow, checkpoint slow, dataloader slow, filesystem bottleneck, FSx throughput, cross-AZ latency, topology mismatch.
中文简介正在准备中。当前显示原文说明。