26 skills · sorted by relevance
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Set up and optimize large language model inference, or response generation, on NVIDIA GPUs. Use quantization, batch processing, and multiple GPUs to improve serving performance, and expose models through a chat completion API.
Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.
Otimiza inferência de LLM com NVIDIA TensorRT para máxima vazão e latência mínima. Use para implantação em produção em GPUs NVIDIA (A100/H100), quando você precisa de inferência 10-100x mais rápida que PyTorch, ou para servir modelos com quantização (FP8/INT4), batching em voo e escalabilidade multi-GPU.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Build visual AI systems that detect objects or divide images into labeled regions. This skill guides dataset preparation, model selection, training, evaluation, inference optimization, and deployment using frameworks such as PyTorch and Ultralytics.
NVIDIA DeepStream SDK development guidance for Python pyservicemaker pipelines, video analytics, TensorRT integration, and streaming inference workflows.
The English summary is being prepared. Showing the original description.
NVIDIA DeepStream model-import guidance for bringing vision models from Hugging Face or NVIDIA NGC into DeepStream pipelines with export, TensorRT build, and benchmark steps.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Find skills for tasks involving NVIDIA products or GPU computing using the live catalog. Get up to three relevant recommendations, with reasons, installation commands, and starter prompts; installation requires explicit approval.
Use for NVIDIA-related requests where an NVIDIA skill might help, even if the user did not ask for a skill. Trigger on NVIDIA products, hardware, software, SDKs, GPUs, Jetson/JetPack/L4T/BSP/SDK Manager/driver/flashing/setup, CUDA, NIM, NeMo, Omniverse/OpenUSD/SimReady, Isaac, RAPIDS/cuDF, cuPyNumeric, cuOpt, Dynamo, Holoscan, TensorRT, DeepStream, VSS, TAO, NGC/NVCF. Do not use for generic non-NVIDIA route, optimize, deploy, AI, video, data, or infrastructure tasks.
The English summary is being prepared. Showing the original description.
Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.
The English summary is being prepared. Showing the original description.
Install or repair the FoundationPose perception pipeline and build its FoundationStereo TensorRT engines. Use for SAM3/TAO dependency conflicts, CUDA library failures, and depth-engine shape or precision decisions.
The English summary is being prepared. Showing the original description.
Use this skill to bring a supported object-detection vision model from HuggingFace or NVIDIA NGC into an NVIDIA DeepStream pipeline with end-to-end automation: ONNX download, SafeTensors export, TRT engine build, custom nvinfer bbox parser, multi-stream benchmark, and PDF report. Object detection models only.
The English summary is being prepared. Showing the original description.
NVIDIA DeepStream SDK development with Python pyservicemaker API. Use when building video analytics pipelines, GStreamer-based video processing, TensorRT inference integration, object detection/tracking, or Kafka/message broker integration.
The English summary is being prepared. Showing the original description.
Build machine vision inspection systems with MATLAB Visual Inspection Toolbox. Covers the full arc from problem selection through deployment: anomaly detection (Student-Teacher, PatchCore, FastFlow, FCDD), object detection (YOLOX), counting (CounTR, shape matching), gauging and measurement (caliper, fitcircle, fitangle), blob analysis (regionprops, calibrated measurement), shape matching for fixturing and localization, synthetic data generation for training augmentation, and deployment to standalone apps, embedded C/C++, NVIDIA GPUs (TensorRT), NPUs (ONNX), and Beckhoff PLCs. Routes to rule-based tools when problems are geometric and deterministic, AI-based tools when defect appearance varies, and hybrid pipelines combining both.
The English summary is being prepared. Showing the original description.
Computer vision engineering for object detection, segmentation, and visual AI, covering CNN and Vision Transformer architectures and ONNX/TensorRT deployment. Use when building detection pipelines, training models, or optimizing inference.
The English summary is being prepared. Showing the original description.
LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns.
The English summary is being prepared. Showing the original description.
Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
The English summary is being prepared. Showing the original description.
Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
The English summary is being prepared. Showing the original description.
优化实际模型推理链路，将正确性对齐、分段 profiling、显存与数据搬运、TensorRT/ONNX/PyTorch 后端、attention/kernel、FP8/compile、缓存与少步采样、质量回归、GPU 成本和服务验收串成同一实验闭环。当用户要求推理提速、降低显存或 GPU 成本、复现模型效果、定位 GPU 利用率低、优化图像/视频/扩散模型或自托管 LLM 时使用，提供 model inference performance optimization 的脱敏案例。纯 API 调用、提示词或 token 用量优化，以及电脑卡顿，不进入本技能的模型推理实验流程。
The English summary is being prepared. Showing the original description.
Use when reducing model size, improving inference speed, or deploying to edge devices - covers quantization, pruning, knowledge distillation, ONNX export, and TensorRT optimizationUse when ", " mentioned.
The English summary is being prepared. Showing the original description.