66건 · 관련도순
Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use with the dstack skill for model-serving work when the image, serving command, resources, backend/fleet choice, or service behavior is not proven. Guides task-first prototyping on real hardware, choosing fleets/backends that can reuse idle instances and caches, checking vLLM/SGLang sources, and verifying the final dstack service with a model request.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Replay an LLM inference request trace (Mooncake / vLLM / SGLang hash_ids format) against a block-level KV prefix cache and compute hit statistics. Use when given a request trace plus cache configuration and asked for hit rate, hit tokens, or final cache contents. Covers the longest-contiguous-prefix semantics that distinguishes KV prefix caching from full-prompt prompt caching, the policy-specific residency and eviction rules (LRU, LFU, S3FIFO), and the partial-last-block accounting rule.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Run all reference benchmarks under benchmarks/eval/ and refresh the reference-table cells in benchmark_*.py. Auto-detects host hardware (H200/H100/H800/...). For each row's matching hw entry: replace cells in place. For new hardware or a new workload tag: append a new row to the section's table. Local-Pipeline-Result tables are skipped. One slash command runs everything end-to-end and auto-commits.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Generate a bounded task plan (and the agent prompt for it) that runs the 5-layer profiling methodology in .claude/skills/model-profiling/METHODOLOGY.md against one model, stops for human confirmation before any GPU work starts, then delegates the actual run to a background agent, verifies its completion claim, and routes findings into a gitignored .profiling-runs/<model>/ directory — durably tracked as a sub-issue under the tracking issue named in SKILL.md's "Result tracking" section — plus the methodology doc.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when evaluating a newly adapted model's accuracy/eval score. Runs evalscope eval (or the test/srt/run_eval.py fallback) against an OpenAI-compatible endpoint on datasets like gsm8k / aime, producing per-dataset reports and a score. Accuracy only — throughput/latency benchmarks belong to model-speed-benchmark. Triggers on accuracy, eval, evalscope, gsm8k, aime, run_eval — not throughput.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Set up the Graphsignal Profiler for inference workloads — vLLM, SGLang, PyTorch, and dstack services. Use when the user wants GPU profiling, tracing, or monitoring for inference, asks about `graphsignal-run` or `graphsignal.watch()`, or asks about CUPTI / Prometheus / OTLP setup.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Think and work like an expert Machine Learning Engineer. Use when a task calls for Machine Learning Engineer judgment. Reasons from decision-policy framing, Google's Rules of ML, point-in-time data, and prefill/decode inference physics through GBDT/PyTorch baselines, vLLM/SGLang/KServe serving with FP8/AWQ quantization, hybrid-retrieval RAG, RAGAS and human-validated LLM judges, OpenTelemetry GenAI tracing, and post-Omnibus EU AI Act obligations while treating train-serve skew, temporal leakage, prompt injection, LLM nondeterminism, and degenerate feedback loops as first-class failure modes.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
LLM serving for latency, batching, caching, quantization, and routing. Use when tuning vLLM or SGLang, sizing KV cache, adding speculative decoding, or cutting serving cost.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when diagnosing NPU + PyTorch or LLM Serving Host CPU affinity, NUMA locality, cgroup/cpuset constraints, CPU range conflicts, PyTorch/runtime threading, DataLoader, tokenizer, scheduler, vLLM-Ascend, SGLang, TTFT, TPOT, tokens/s, QPS, or multi-rank/multi-worker CPU binding issues.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
DB格式Profiling数据场景化性能拆解：自动完成场景确认（框架/任务类型判定）、阶段拆解与单次执行边界拆解，并给出报告。Use when 需要对昇腾 Profiling DB 做阶段拆解、step 归因或生成性能报告, or when the user says "帮我按任务阶段拆解这个Profiling数据", "多层次性能耗时拆解", "分析这个 vllm/sglang/verl 数据，主要耗时都花费在哪里了"。
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Fornece orientação para treinamento RL de nível empresarial usando miles, um fork pronto para produção do slime. Use ao treinar grandes modelos MoE com FP8/INT4, necessitando alinhamento treino-inferência ou exigindo RL especulativo para máxima taxa de transferência.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says "deploy this LLM", "host this HuggingFace model", "serve this fine-tuned model", "deploy this embedding model", "host a reranker", "serve a sentence-transformers model", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings/cross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.