45 skills · sorted by relevance
Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.
Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.
Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.
Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
优化实际模型推理链路，将正确性对齐、分段 profiling、显存与数据搬运、TensorRT/ONNX/PyTorch 后端、attention/kernel、FP8/compile、缓存与少步采样、质量回归、GPU 成本和服务验收串成同一实验闭环。当用户要求推理提速、降低显存或 GPU 成本、复现模型效果、定位 GPU 利用率低、优化图像/视频/扩散模型或自托管 LLM 时使用，提供 model inference performance optimization 的脱敏案例。纯 API 调用、提示词或 token 用量优化，以及电脑卡顿，不进入本技能的模型推理实验流程。
The English summary is being prepared. Showing the original description.
Think and work like an expert Machine Learning Engineer. Use when a task calls for Machine Learning Engineer judgment. Reasons from decision-policy framing, Google's Rules of ML, point-in-time data, and prefill/decode inference physics through GBDT/PyTorch baselines, vLLM/SGLang/KServe serving with FP8/AWQ quantization, hybrid-retrieval RAG, RAGAS and human-validated LLM judges, OpenTelemetry GenAI tracing, and post-Omnibus EU AI Act obligations while treating train-serve skew, temporal leakage, prompt injection, LLM nondeterminism, and degenerate feedback loops as first-class failure modes.
The English summary is being prepared. Showing the original description.
Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.
The English summary is being prepared. Showing the original description.
对 DiT（扩散/多模态生成）模型执行 W8A8 动态（data-free）或 W8A8 MXFP8（data-free）量化。覆盖 HunyuanVideo、Wan2.2-T2V/I2V/TI2V、FLUX.1-dev（已迁移）、SD3、Sana、HunyuanDiT、CogViewX 等。YAML apiversion 固定 multimodal_sd_modelslim_v1，统一 MultimodalPipelineInterface 重构路径（不再保留 LegacyMultimodalPipelineInterface）。模板与字段以 msmodelslim/model/hunyuan_video/model_adapter.py（唯一已迁移到重构路径且 lab_practice YAML 已 validated 的 DiT）为参考；其它 DiT 需按各自推理仓的 `parse_args` 调整字段集合。data-free 指不引入外部激活校准数据，spec.dataset（校准集短名，短名如 wan2_2_t2v；enable_dump: false 时不参与 dump）仍必填；精度调优通过下游经验库 L2 §7 + 直接调 `quantization-expert-experience-tuning-rules/scripts/apply_rollback.py`（整层回退）扩展 exclude 列表，闭环由 `quantization-accuracy-tuning-orchestrator` 调度。
The English summary is being prepared. Showing the original description.
为 msModelSlim 适配流程注入反量化能力。先识别模型权重是否可反量化，再实现反量化脚本并接入 model_adapter。当前仅覆盖 FP8 的 per-block 与 per-channel 两类；若格式不确定或无公开反量化规则，要求用户提供反量化脚本或浮点权重。
The English summary is being prepared. Showing the original description.
Iterate on a GenAI notebook against the self-hosted stack via the genai-stack CLI (config dirs, auth, subdomains, quantization, GPU/VRAM). Arguments: <notebook|service> [--service comfyui|forge|vllm] [--quant int4|fp8] [--validate] [--bg]
The English summary is being prepared. Showing the original description.
DiT 计算模块（L3）：把**已定位的 DiT 计算瓶颈**落成特性级选档与实施—— 量化档（W8A16 / W4A16 / W8A8 系列 / W4A4 / MXFP8 / FA 量化）、稀疏（rf_v2 / ada_bsa）、 缓存（DiTCache / AttentionCache / 时间步优化）、编译启用（MindieSDBackend / Pattern 融合 / ACLGraph） 的**开不开、开哪一档、怎么开、怎么复验**；依据是 `docs/zh/features/*`（特性真源）+ framework-integration/references/framework-support-matrix.md（支持状态）。 即使用户只说"这个模型怎么加速""量化/稀疏/Cache 怎么选怎么开""要不要开量化、开哪一档""这个档位开了有没有效果" 而未提 profiling，也应触发。 **入口条件**：瓶颈点已明确（用户带一句实测锚点，或编排层交付标签）时由域入口 `performance-optimization` 按标签分发到本技能；**瓶颈未明（"怎么加速 / 跑通 / 采 profile"）先走 `model-auto-optimization` 定位**，不在本技能内做占比分析。 near-miss：多卡并行形态 / 通信掩盖 / TP·offload 选型 → `dit-parallel-opt`；VAE 解码段与 host 固定开销 → 各自模块（VAE / host）；单算子实现级实测选型（mindie_bench）→ `benchmark-dev`； 需要新增 pattern / 算子才能落地本档 → `pattern-dev` / `operator-dev`；框架侧开关与使能验证 → `framework-integration`；量化器位级契约与精度对齐（编码公式 / 舍入 / scale 粒度）→ `quantization-dev`；精度验收判据 → `accuracy-gate`；数字入库口径 → `perf-gate`。
The English summary is being prepared. Showing the original description.
Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.
The English summary is being prepared. Showing the original description.
API de treinamento distribuído mais simples. 4 linhas para adicionar suporte distribuído a qualquer script PyTorch. API unificada para DeepSpeed/FSDP/Megatron/DDP. Posicionamento automático de device, precisão mista (FP16/BF16/FP8). Config interativo, comando de launch único. Padrão do ecossistema HuggingFace.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
The English summary is being prepared. Showing the original description.
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
The English summary is being prepared. Showing the original description.