12 skills · sorted by relevance
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Set up and optimize large language model inference, or response generation, on NVIDIA GPUs. Use quantization, batch processing, and multiple GPUs to improve serving performance, and expose models through a chat completion API.
Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.
Otimiza inferência de LLM com NVIDIA TensorRT para máxima vazão e latência mínima. Use para implantação em produção em GPUs NVIDIA (A100/H100), quando você precisa de inferência 10-100x mais rápida que PyTorch, ou para servir modelos com quantização (FP8/INT4), batching em voo e escalabilidade multi-GPU.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns.
The English summary is being prepared. Showing the original description.
Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
The English summary is being prepared. Showing the original description.
Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
The English summary is being prepared. Showing the original description.
NVIDIA DGX Spark User Guide リファレンス。GB10 Grace Blackwell Superchip 仕様、 first-boot / UEFI、ConnectX-7 Networking (旧 Spark Stacking)、複数 Spark 接続、 NVIDIA Sync Cluster Assistant、DGX OS / NGC / container runtime、 Release Notes (DGX OS 7.5.0)、known issues、PXE / cloud-init、fleet 運用、 apt / fwupdmgr、recovery、PSIRT、 NVIDIA Developer Forums 由来 troubleshooting、 build.nvidia.com/spark playbook (vLLM / vLLM cluster / TensorRT-LLM / Ollama / ComfyUI / NIM / NeMo / VLM fine-tuning / NVIDIA PAIR / NCCL)、Tailscale、VS Code。
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.