'throughput' 검색 결과

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Interactively gather parameters for `python -m cli.inference.throughput_optimizer`, generate a deployment simulation command, explain assumptions, ask for execution confirmation, then run the simulation and summarize the best parallel strategy. Use when the user wants to evaluate a model on one or more hardware profiles, compare hardware targets, compare aggregation versus disaggregation versus PD ratio optimization, choose TP/EP/MOE-DP search ranges, or obtain recommended concurrency, throughput, TTFT, and TPOT results from throughput modeling.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Run concurrent Omnigent conversations with real hosts and runners while replacing the LLM with a zero-latency mock. Read the generated report to explain response-time distributions, throughput, and failures, helping assess Omnigent's own processing overhead.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Measuring performance with concurrent hosts
• Studying latency as conversation history grows
• Comparing throughput and failures under load

Analyze and optimize system throughput including request handling, data processing, and resource utilization. Use when identifying capacity limits or evaluating scaling strategies. Trigger with phrases like "analyze throughput", "optimize capacity", or "identify bottlenecks".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

현재 성능 측정, 가설별 변경안 비교, 정확성 검증으로 최적화를 진행합니다. 시간과 비용 한도 안에서 가장 빠르고 안전한 안을 고르고 재실행 절차를 남깁니다.

• 느린 작업을 측정하며 속도 개선
• 같은 입력 조건에서 구현안 비교
• 응답 시간, 처리량, 실행 비용 평가

vLLM으로 대규모 언어 모델을 API로 제공하거나 많은 입력을 일괄 처리하도록 돕습니다. 배포와 성능 모니터링, 메모리 절약, 응답 속도 조정도 다룹니다.

• OpenAI 호환 API를 제공할 때
• 많은 프롬프트를 일괄 처리할 때
• 제한된 GPU 메모리로 모델을 실행할 때

Build Python code for Azure Cosmos DB's NoSQL API to create, read, update, delete, and query documents. Covers authentication, containers for storing data, partition keys for distributing records, throughput settings, asynchronous operations, and error handling.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Implementing document operations in Python
• Writing parameterized queries for filtered searches
• Configuring containers, partition keys, and throughput

Check ten recurring problems before or during machine learning runs on NVIDIA DGX Spark. Diagnose startup failures, out-of-memory errors, and falling throughput, with guidance on compatible environments, resource use, numeric formats, and training across two devices.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Checking a DGX Spark before a long training run
• Diagnosing memory errors despite apparent free capacity
• Investigating throughput drops during training

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Investigate slow TPU training on GKE by comparing diagnostic events with one-minute monitoring metrics. Separate memory or CPU bottlenecks from hardware and network issues, and identify the appropriate next steps.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating slower TPU training without pod crashes
• Checking PERFORMANCE_DEGRADATION events
• Separating resource bottlenecks from infrastructure issues

Investigate Google Cloud network problems using traffic logs, performance metrics, and connection path tests. Identify blocked traffic, packet loss, and latency, summarize traffic volumes, or estimate VPC Flow Logs costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate blocked traffic and packet loss
• Check latency and throughput trends
• Analyze Cloud NAT port exhaustion

Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Diagnosing slow training and GPU memory shortages
• Investigating loss spikes and invalid numerical values
• Debugging communication hangs across GPUs and machines

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Guides PyTorch model training and fine-tuning, from choosing architectures to configuring optimizers and learning rates. Provides troubleshooting steps for unstable loss and memory shortages, plus methods for improving GPU throughput and comparing experiments.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose model architectures for the available data type and scale
• Configure optimizers and learning-rate schedules
• Investigate loss spikes, NaNs, and GPU memory shortages

Plan how to split AI models and workloads across multiple GPUs. Compare memory needs, computation, communication, and waiting time to estimate completion times and choose a layout suited to response time, throughput, or cost goals.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running models that exceed a single GPU's memory
• Comparing ways to split work across GPUs
• Estimating communication costs within and between servers

Run `doca_bench` (DOCA 2.7.0 or newer) to measure throughput, bulk latency, precision latency, or maximum bandwidth for RDMA, Compress, AES-GCM, SHA, DMA, EC, Ethernet, Comch, or GPUNetIO on a host or BlueField Arm. Use it to discover enabled benchmark libraries, capture a reproducible command/version/device/environment baseline, compare stable runs against a declared tolerance, or diagnose configuration, device-binding, workload-precondition, and measurement failures. Trigger for requests such as measuring BlueField compression speed, NIC RDMA throughput, crypto latency, or a pre-upgrade baseline. Do not use for application end-to-end timing, custom benchmark code, DOCA installation, or binary patches.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use this skill when the user is building, running, or interpreting the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA kernel on the client posts RDMA WRITE work requests through the doca-gpunetio device-side surface to measure sustained GPU-driven WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user does not explicitly mention "doca-gpunetio-ib-write-bw" or "GPUNetIO" — typical implicit phrasings include "measure WRITE BW when the GPU posts the WRs", "BW swings between runs on the same flags", "is the NIC saturated or am I CPU-bound on the CUDA kernel", "meson compile fails for the GPUNetIO bw tool", "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated WRITE throughput vs CPU-initiated perftest". Refuse and route elsewhere for general doca-gpunetio library work, DOCA install, the GPU-initiated WRITE latency analog, the CPU-initiated upstream perftest, or application-level end-to-end throughput — those belong to other skills.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

[omh] AI agent progress for managers: help managers inspect AI-agent progress, blockers, quality gates, and throughput levers. Use when the user says: agent-ops-review, agent ops review, agent productivity, operator productivity, manager view, quality dashboard, throughput review, agent work quality.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design account-aware Alchemy throttling, bounded retries, and admission control from observed throughput rather than stale plan constants. Use when preventing or handling rate limits. Trigger with "Alchemy rate limit", "Alchemy throughput", or "fix Alchemy 429s".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Run the measurement and ceremony layer of a delivery team: the metric set, estimation and capacity, the five sprint ceremonies, and the sprint report and quarterly rollup. Use this skill whenever the user is acting as PMO or scrum lead, planning a sprint, running standup, refinement, review, or retro, measuring velocity, cycle time, or throughput, checking capacity, or reporting on how a team is doing. Triggers on PMO, sprint planning, plan the sprint, standup, refinement, backlog grooming, sprint review, retro, velocity, capacity, cycle time, lead time, throughput, WIP, burndown, carryover, sprint report, story points to hours, how is the team doing, delivery metrics, agent lanes. Also triggers when the user asks how much work fits in a sprint or whether a team is on track, even if they never say sprint or PMO.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.