'vllm' 검색 결과

Prepare GRPO reinforcement learning configurations for tasks with programmatically checkable answers, such as math, code, and structured output. Design and inspect reward functions, then select training variants when runs show instability or exploit flaws in scoring.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing GRPO training for tasks with automatic correctness checks
• Designing rewards for output format and correctness
• Inspecting reward scores before training

Analyze LLM serving traces to identify GPU time usage, opportunities to run operations concurrently, and known patterns for combining operations. Produce three consistent tables from saved traces or live captures on supported servers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Identify GPU operations that dominate inference time
• Compare input processing and token generation
• Trace GPU operations back to Python code

Use Outlines to make language models produce data in a defined format. The skill covers extracting information into JSON, restricting classification labels, and applying Pydantic models, which define fields and data types, with local model backends.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting company or profile details into JSON
• Classifying text using a fixed set of labels
• Generating text that matches a specified pattern

Build safety checks for LLM prompts and responses using Meta's LlamaGuard moderation model. Classify potentially harmful content, block flagged messages, serve checks through an API, and integrate them with NeMo Guardrails.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Checking prompts before sending them to an LLM
• Filtering responses before displaying them
• Building a moderation API

Create or update Kubernetes YAML configuration files for GKE Autopilot and Standard. Translates application requirements into manifests covering container permissions, CPU and memory allocation, health checks, storage, external routing, and GPU resources for AI model serving.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating GKE application deployment manifests
• Reviewing container security and resource settings
• Configuring external routes with GKE Gateway API

Use Outlines to make language models produce data in a defined format. The skill covers extracting information into JSON, restricting classification labels, and applying Pydantic models, which define fields and data types, with local model backends.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting company or profile details into JSON
• Classifying text using a fixed set of labels
• Generating text that matches a specified pattern

Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Review and upgrade MetaX model support against a target vLLM revision and installed MACA components, recursively including model-dependent attention and kernels. Establish quantization and cache differences, preserve upstream structure, and distinguish shared upstream bugs from adaptation defects. Use for model support work, not standalone monkey-patch or registry audits.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Establish the shared environment, source/runtime correspondence, target confirmation and validation evidence for MetaX vLLM upgrades. Use as the common prerequisite of the patch, attention, registry and model upgrade skills, or for an explicitly requested MetaX environment preflight; it does not perform domain-specific adaptations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

环境安装与准备：把部署环境从零安装就绪——mindiesd 编译安装（本地昇腾直装 / SSH 推远端容器 / Docker 镜像直装）与三方推理框架全栈安装（vLLM-Omni 源码构建、DiffSynth-Engine 部署、 LightX2V editable 部署），并负责模型权重确认与下载（下载前先确认远端是否已存在）。不含特性使能与验证 （framework-integration）与 profiling（profiling-collect）；SSH 工具由 remote-access 提供。 当用户需要安装 MindIE-SD、源码构建/直装 vLLM-Omni 或 LightX2V（editable + PLATFORM=ascend_npu）、 或确认/下载模型权重时使用此技能； 即使用户只提到"把代码推到服务器""在容器里装 vllm 全栈""准备 lightx2v 调优环境"而未说昇腾， 只要上下文涉及环境安装与准备都应触发。由 model-auto-optimization S0 与 dev-workflow 部署阶段指引加载。

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Framework RLHF de alta performance com aceleração Ray+vLLM. Use para treinamento PPO, GRPO, RLOO, DPO de modelos grandes (7B-70B+). Construído em Ray, vLLM, ZeRO-3. 2× mais rápido que DeepSpeedChat com arquitetura distribuída e compartilhamento de recursos GPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serve LLMs com alta throughput usando PagedAttention do vLLM e continuous batching. Use ao fazer deploy de APIs LLM em produção, otimizar latência/throughput de inferência, ou servir modelos com memória GPU limitada. Suporta endpoints compatíveis com OpenAI, quantização (GPTQ/AWQ/FP8) e tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Pick the right serving container for a SageMaker model deployment and find its current image URI. Use this skill whenever about to deploy a model to a SageMaker endpoint and an image URI needs to be chosen — including when the user says "deploy this LLM", "host this HuggingFace model", "serve this fine-tuned model", "deploy this embedding model", "host a reranker", "serve a sentence-transformers model", or when about to hardcode any container URI in deployment code. HuggingFace-curated Deep Learning Containers are ALWAYS preferred: HuggingFace vLLM (LLMs and generative rerankers), HuggingFace vLLM-Omni (multimodal), TEI (embeddings/cross-encoder rerankers), HF Inference Toolkit (other transformers). Generic images (AWS vLLM, DJL-LMI, SGLang) are used only when no HuggingFace image is compatible — never merely because they carry a newer version. Never hardcode a container URI from memory and never default to TGI. Prevents stale-image failures and wrong-region URIs.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

High-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Set up ruvector@0.3.3 with supporting packages, register its MCP server for agent access, and verify installation with diagnostic commands and smoke tests. Helps prepare the dependencies needed for initial use of /vector subcommands.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up ruvector for the first time
• Resolving missing dependencies for Brain, SONA, or LLM commands
• Registering the ruvector MCP server with Claude

Estimate the cost per million tokens, units of model text, on hardware you own using power consumption, electricity rates, hardware costs, and measured generation speed. Compare utilization levels to find when local operation costs less than a hosted model.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare local model costs with a hosted API
• Estimate costs at different utilization levels
• Include electricity and hardware depreciation in costs

Export an approved fine-tuned model into a format suited to its hardware and runtime. Choose weight packaging and quantization, which reduces numerical precision, then compare outputs before and after export to check for regressions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Exporting a checkpoint approved for deployment
• Choosing a model format for a target GPU or laptop
• Comparing outputs before and after export

Set up reinforcement learning, which trains models using rewards, with verl. Follow workflows for preparing data, configuring GRPO or PPO training, scaling across GPUs, and addressing memory errors or unstable training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Training a math reasoning model with GRPO
• Configuring PPO training with a critic model
• Scaling large model training across multiple nodes

Set up model training with torchforge for reinforcement learning, where rewards guide improvement. Covers training configurations, custom reward and loss functions, and distributed execution with Monarch and TorchTitan, keeping algorithm experiments separate from infrastructure concerns.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train mathematical reasoning models with GRPO
• Experiment with custom reward and loss functions
• Scale training across multiple GPUs or nodes

Guides migration of existing AI inference workloads to self-hosted GKE, Google’s managed Kubernetes service. Covers requirements, GPU sizing, deployment configuration, model storage, validation, and traffic cutover using gcloud and kubectl.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Migrating an existing Cloud Run AI service to GKE
• Planning a self-hosted replacement for Gemini API inference
• Sizing GPUs and preparing deployment configuration

Guides LLM training with verl using rewards to improve model responses. Covers data preparation, reward functions, GRPO and PPO configuration, distributed GPU training, and troubleshooting memory shortages or unstable training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train math reasoning models with GRPO and answer-based rewards
• Configure PPO training with a separate value-estimation model
• Run large-model training across multiple GPU nodes

Guides reinforcement learning, where models learn from rewards, using torchforge and PyTorch. Covers reward design, custom loss functions, and distributed training so researchers can experiment with algorithms while keeping infrastructure concerns separate.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train mathematical reasoning models with GRPO
• Experiment with custom loss functions
• Scale training across multiple GPUs or nodes

Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.