68건 · 관련도순
Compare batching, request scheduling, and memory management for a single model serving instance. Assess concurrent request capacity, response times, generation speed, and GPU time per result that meets the required quality and deadline.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Build chat, image generation, and speech transcription features on 0G Compute Network, a distributed GPU marketplace. This skill guides SDK integration, model fine-tuning, response processing for billing, and account funding through code patterns and CLI commands.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Deploy ML models on Kubernetes with KServe (formerly KFServing) and NVIDIA Triton Inference Server. Includes canary deployments, autoscaling, model versioning, A/B testing, and GPU resource management for production model serving.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Design and configure model serving infrastructure — endpoint scaling, batching, GPU allocation. Use when asked to "serve this model", "design an inference endpoint", or "size GPU allocation for serving".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Build a feature store using Feast for centralized feature management, configure offline and online stores for batch and real-time serving, define feature views with transformations, and implement point-in-time correct joins for ML pipelines. Use when managing features for multiple ML models, ensuring training-serving consistency, serving low-latency features for real-time inference, reusing feature definitions across projects, or building a feature catalog for discovery and governance.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Geração estruturada rápida e serving para LLMs com RadixAttention para cache de prefixos. Use para saídas JSON/regex, decodificação restrita, workflows com agentes que fazem chamadas de ferramentas, ou quando você precisa de 5× mais rápido que vLLM com compartilhamento de prefixos. Alimenta 300.000+ GPUs na xAI, AMD, NVIDIA e LinkedIn.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says "deploy this LLM", "host this HuggingFace model", "serve this fine-tuned model", "deploy this embedding model", "host a reranker", "serve a sentence-transformers model", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings/cross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like "deploy a model", "host this LLM on AWS", "serve this embedding model", "deploy a reranker", "deploy a text-to-image / diffusion model", "host this for async inference", "create an endpoint", "serve my fine-tuned model", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. "I just want to get this running on AWS, you figure it out"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image / diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Create or update Kubernetes YAML configuration files for GKE Autopilot and Standard. Translates application requirements into manifests covering container permissions, CPU and memory allocation, health checks, storage, external routing, and GPU resources for AI model serving.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Implements the NOWAIT technique for efficient reasoning in R1-style LLMs. Use when optimizing inference of reasoning models (QwQ, DeepSeek-R1, Phi4-Reasoning, Qwen3, Kimi-VL, QvQ), reducing chain-of-thought token usage by 27-51% while preserving accuracy. Triggers on "optimize reasoning", "reduce thinking tokens", "efficient inference", "suppress reflection tokens", or when working with verbose CoT outputs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
LLM serving for latency, batching, caching, quantization, and routing. Use when tuning vLLM or SGLang, sizing KV cache, adding speculative decoding, or cutting serving cost.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
MLOps and the production ML lifecycle -- model packaging and serving, CI/CD for ML, experiment tracking, model registries, reproducibility, production monitoring for data and concept drift, retraining pipelines, A/B and shadow deployment, and rollback. Covers batch vs online/real-time inference, REST endpoints, feature stores, data and version pinning, deterministic pipelines, performance-decay detection, and retraining triggers. Use when deploying models to production, serving predictions, monitoring for data or concept drift, setting up ML CI/CD, tracking experiments, managing a model registry, or planning retraining, shadow rollout, and rollback.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Executa inferência de LLM em CPU, Apple Silicon e GPUs consumer sem hardware NVIDIA. Use para edge deployment, Macs M1/M2/M3, GPUs AMD/Intel ou quando CUDA não está disponível. Suporta quantização GGUF (1,5-8 bits) para redução de memória e aceleração de 4-10× vs PyTorch em CPU.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Otimiza inferência de LLM com NVIDIA TensorRT para máxima vazão e latência mínima. Use para implantação em produção em GPUs NVIDIA (A100/H100), quando você precisa de inferência 10-100x mais rápida que PyTorch, ou para servir modelos com quantização (FP8/INT4), batching em voo e escalabilidade multi-GPU.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when working with Bentoml — bentoML model serving and packaging management. Covers service management, model packaging, deployment status, API testing, Bento building, and runner configuration. Use when packaging ML models for serving, deploying BentoML services, debugging inference issues, or managing model artifacts.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Present results returned by a Codex helper while preserving findings, severity, file references, and uncertainty. This internal guidance also covers reporting edited files, failed runs, and setup requirements, with user confirmation before fixing review findings.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Estimate how model architecture, precision, and context length affect weight storage, generation state, computation, and memory traffic. Compare models and attention designs with separate resource estimates for input processing and token generation.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.