111 件 ・ 関連度順
概要と使いどころ
Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
日本語の概要は準備中です。原文の説明を表示しています。
Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
日本語の概要は準備中です。原文の説明を表示しています。
Query Finelog logs and telemetry for Iris tasks, workers, profiles, training, vLLM, and cross-cluster forwarding. Use for schema discovery, SQL, memory or CPU summaries, hub-versus-regional comparisons, counter semantics, and query-performance diagnosis.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Create, refine, or fix NVIDIA voice agents (Cascaded or Omni) with Pipecat or LiveKit. Use when building, scaffolding, or iterating on a real-time voice-agent pipeline, including speech (ASR/TTS) customization and cloud or local deployment. Not for offline/batch speech-to-text, text-only chat or RAG, or generic Docker or CUDA work unrelated to a voice agent.
日本語の概要は準備中です。原文の説明を表示しています。
Pick Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes; maps Orin SM 8.7 vs Thor SM 11.0 and JetPack-specific package choices.
日本語の概要は準備中です。原文の説明を表示しています。
Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottleneck.
日本語の概要は準備中です。原文の説明を表示しています。
Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.
日本語の概要は準備中です。原文の説明を表示しています。
Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.
日本語の概要は準備中です。原文の説明を表示しています。
Use this skill when building, deploying, evaluating, debugging, or measuring latency for the DeepStream SOP Inference Microservice — a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection (GEBD) plus VLM classification. Trigger even if the user does not name it: verify operator step sequence, detect missing or out-of-order SOP steps, score factory/work-cell video for procedure compliance, run VLM-based SOP checking on industrial cameras, or call /v1/chat/completions with a file, RTSP, or Basler camera. Also trigger for its internals: SOPVideoProcessor, DeepStream GEBD model (e.g. DDM) via Triton CAPI, nvds_custom_postprocess, Cosmos Reason 1/2 vLLM, SSE streaming, Kafka NvProto/JSON output, Basler/Pylon camera + emulation, Docker compose, chunk-level latency. Do NOT trigger for generic DeepStream pipelines, object detection/tracking, NIM imports, or video summarization.
日本語の概要は準備中です。原文の説明を表示しています。
[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the endpoint with the standard TTFT/TPOT/goodput protocol. Use when the user says: inference-serving, inference serving, serve this model, serve the model, model serving, serving endpoint, vllm, llama.cpp.
日本語の概要は準備中です。原文の説明を表示しています。
Set up AI Runway on AKS — from bare cluster to running model. Covers cluster verification, controller install, GPU assessment, provider setup, and first deployment. WHEN: "setup AI Runway", "onboard AKS cluster", "install AI Runway", "airunway setup", "deploy model to AKS", "GPU inference on AKS", "KAITO setup on AKS", "run LLM on AKS", "vLLM on AKS", "set up model serving on AKS", "AI Runway controller".
日本語の概要は準備中です。原文の説明を表示しています。
Delegate a coding task to Aider (`aider`) as a background implementer, then review its diff and land it yourself. Use this whenever the user wants to hand implementation work to Aider - phrasings like "have Aider do X", "delegate this to aider", "run it through Aider", or "use Aider to implement/fix/refactor" - or wants to run a queue of coding tasks through Aider while staying the reviewer. This includes asking Aider to drive a local or self-hosted OpenAI-compatible endpoint ("have Aider use my local model", "run Aider against llama.cpp / Ollama / vLLM / LM Studio"), which Aider reaches via `--api-base`. DO NOT USE for local-model or coding requests that do not name Aider, for tasks small enough to do inline, or when the user wants the code written directly without delegating.
日本語の概要は準備中です。原文の説明を表示しています。
Use with the dstack skill for model-serving work when the image, serving command, resources, backend/fleet choice, or service behavior is not proven. Guides task-first prototyping on real hardware, choosing fleets/backends that can reuse idle instances and caches, checking vLLM/SGLang sources, and verifying the final dstack service with a model request.
日本語の概要は準備中です。原文の説明を表示しています。
Add file-based observability, tracing, and run analysis to LLM/agent experiments — per-turn transcripts, tool calls and results, reasoning traces, token/cost accounting, per-episode artifacts, plus terminal viewers and a run-comparison report. Platform-agnostic: one small adapter covers OpenAI, Gemini/Vertex, Anthropic, LiteLLM, Tinker, vLLM, or any custom inference stack, and the same tooling then works unchanged. Use this skill whenever the user wants observability, tracing, logging, or run tracking for model experiments; asks why an agent failed a task or wants to replay/debug/diff an episode; wants to compare models, arms, or ablations; wants to instrument a new folder, repo, or platform ("add observability here", "do the same for Tinker/Gemini/OpenAI"); wants to import or tie in existing experiment results so old runs show up alongside new ones; or wants eval runner scripts where many models share one protocol. Reach for this even when they only say "log what the model is doing", "I can't tell what happened in this run", or "set up traces for this experiment".
日本語の概要は準備中です。原文の説明を表示しています。
Replay an LLM inference request trace (Mooncake / vLLM / SGLang hash_ids format) against a block-level KV prefix cache and compute hit statistics. Use when given a request trace plus cache configuration and asked for hit rate, hit tokens, or final cache contents. Covers the longest-contiguous-prefix semantics that distinguishes KV prefix caching from full-prompt prompt caching, the policy-specific residency and eviction rules (LRU, LFU, S3FIFO), and the partial-last-block accounting rule.
日本語の概要は準備中です。原文の説明を表示しています。
Set up AI Runway on AKS — from bare cluster to running model. Covers cluster verification, controller install, GPU assessment, provider setup, and first deployment. WHEN: "setup AI Runway", "onboard AKS cluster", "install AI Runway", "airunway setup", "deploy model to AKS", "GPU inference on AKS", "KAITO setup on AKS", "run LLM on AKS", "vLLM on AKS", "set up model serving on AKS", "AI Runway controller".
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
日本語の概要は準備中です。原文の説明を表示しています。
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
日本語の概要は準備中です。原文の説明を表示しています。
LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns.
日本語の概要は準備中です。原文の説明を表示しています。
Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
日本語の概要は準備中です。原文の説明を表示しています。