12 个 · 按相关度排序
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
中文简介正在准备中。当前显示英语简介。
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
中文简介正在准备中。当前显示英语简介。
帮助在NVIDIA GPU上加速大语言模型的响应生成，并通过API提供服务。支持量化、批量处理和多GPU分布运行，以调整显存占用与服务性能。
Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.
中文简介正在准备中。当前显示英语简介。
Otimiza inferência de LLM com NVIDIA TensorRT para máxima vazão e latência mínima. Use para implantação em produção em GPUs NVIDIA (A100/H100), quando você precisa de inferência 10-100x mais rápida que PyTorch, ou para servir modelos com quantização (FP8/INT4), batching em voo e escalabilidade multi-GPU.
中文简介正在准备中。当前显示原文说明。
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
中文简介正在准备中。当前显示原文说明。
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
中文简介正在准备中。当前显示原文说明。
LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns.
中文简介正在准备中。当前显示原文说明。
Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
中文简介正在准备中。当前显示原文说明。
Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
中文简介正在准备中。当前显示原文说明。
NVIDIA DGX Spark User Guide リファレンス。GB10 Grace Blackwell Superchip 仕様、 first-boot / UEFI、ConnectX-7 Networking (旧 Spark Stacking)、複数 Spark 接続、 NVIDIA Sync Cluster Assistant、DGX OS / NGC / container runtime、 Release Notes (DGX OS 7.5.0)、known issues、PXE / cloud-init、fleet 運用、 apt / fwupdmgr、recovery、PSIRT、 NVIDIA Developer Forums 由来 troubleshooting、 build.nvidia.com/spark playbook (vLLM / vLLM cluster / TensorRT-LLM / Ollama / ComfyUI / NIM / NeMo / VLM fine-tuning / NVIDIA PAIR / NCCL)、Tailscale、VS Code。
中文简介正在准备中。当前显示原文说明。
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
中文简介正在准备中。当前显示原文说明。