'vllm' 검색 결과

Neutral, framework-agnostic decision tree for project kickoff: "which agent / RAG / LLM framework should I reach for?" Synthesizes the ecosystem sections of 7 landmark-project SOPs (LangGraph, LlamaIndex, DSPy, CrewAI, vLLM, Aider, Dify) into one layered rubric. Core stance: frameworks are LAYERS, not competitors — a real project usually combines DSPy (compile) + LlamaIndex (retrieve) + LangGraph (orchestrate) + vLLM (serve), and you choose ONE per layer, not one to rule all. Use when starting any LLM/agent/RAG project, or whenever the "which framework?" question is asked. Deliberately neutral — unlike vendor docs and the LangChain-biased `framework-selection` on skill.sh, this skill has no horse in the race.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Trim large MetaX model directories for dummy smoke tests or real-checkpoint loading on limited GPUs. Preserve target execution paths while reducing depth; use the real-weight workflow when a loadable checkpoint is requested. Not for accuracy evaluation or model compatibility upgrades.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

profiling 数据统一采集：远端昇腾采集 NPU 性能数据回传本地，供 profiling-analyze 分析。 两条入口——mindiesd 自家脚本入口（collect_profile.py）与三方框架入口 （collect_patch_template.py 补丁 + torchrun，含 vLLM-Omni HTTP 服务等分场景）； warmup 在 profiler 外（默认 5 步，compile ≥10）。 只要用户想"开 profiler/采 profile/拿算子级数据"，自家脚本或 vllm、LightX2V 等框架都先 用本入口；分析/瓶颈定位/优化选型/并行/基准请求分别属 profiling-analyze、 dit-perf-opt、dit-parallel-opt、benchmark-dev，本入口不承接。 由 model-auto-optimization 的 S1（融合分析）阶段调用，亦由 profiling-analyze 与 dev-workflow 的 采集场景指引加载。

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vLLM: high-throughput LLM serving, OpenAI API, quantization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Garanta estrutura válida de JSON/XML/código durante a geração, use modelos Pydantic para outputs type-safe, suporte modelos locais (Transformers, vLLM) e maximize velocidade de inferência com Outlines - biblioteca de geração estruturada da dottxt.ai

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vLLM: high-throughput LLM serving, OpenAI API, quantization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Guarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local models (Transformers, vLLM), and maximize inference speed with Outlines - dottxt.ai's structured generation library

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

NVIDIA DGX Spark User Guide リファレンス。GB10 Grace Blackwell Superchip 仕様、 first-boot / UEFI、ConnectX-7 Networking (旧 Spark Stacking)、複数 Spark 接続、 NVIDIA Sync Cluster Assistant、DGX OS / NGC / container runtime、 Release Notes (DGX OS 7.5.0)、known issues、PXE / cloud-init、fleet 運用、 apt / fwupdmgr、recovery、PSIRT、 NVIDIA Developer Forums 由来 troubleshooting、 build.nvidia.com/spark playbook (vLLM / vLLM cluster / TensorRT-LLM / Ollama / ComfyUI / NIM / NeMo / VLM fine-tuning / NVIDIA PAIR / NCCL)、Tailscale、VS Code。

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

AI基础设施实战——从GPU集群调度到模型推理部署的全栈实操指南。覆盖：昇腾/GPU统一调度、PyTorch分布式训练、vLLM推理部署、模型量化实战、LLMOps、GPU故障诊断、HuggingFace生态、MLOps流水线。触发词：AI基础设施、GPU、昇腾、训练、推理、vLLM、PyTorch分布式、模型部署、LLMOps、MLOps、集群调度、HuggingFace

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

LLM이 JSON, 정규 표현식, Pydantic 타입 정의에 맞춰 답변을 생성하도록 합니다. 텍스트에서 추출한 정보와 분류 결과를 후속 프로그램에서 다루기 쉬운 형식으로 정리합니다.

• 텍스트에서 기업과 제품 정보를 추출할 때
• 정해진 라벨로 리뷰를 분류할 때
• 프로필을 항목별로 정리할 때

OpenHands CLI에 코드 수정과 여러 단계의 개발 작업을 맡깁니다. LiteLLM 지원 모델을 선택하고, 대화형 화면 없이 일괄 실행과 진행 확인, 대화 재개를 할 수 있습니다.

• OpenHands에 코드 수정 위임
• LiteLLM 지원 모델로 개발
• 오래 걸리는 개발 작업의 진행 확인

Convert chat messages into template strings for local model runtimes such as Ollama or llama.cpp. Optionally route queries among a few registered intents using embeddings, numerical representations of meaning, without calling a model.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing prompt strings for Ollama
• Formatting conversations for a local model
• Routing queries among a few intents

Search document collections by meaning using embeddings, numerical representations of text. Compare text similarity, route queries among a few predefined patterns, and use a compressed search index when memory is limited.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding documents with similar meaning
• Comparing the similarity of two pieces of text
• Reducing memory use for large vector collections

Set up machine learning training and inference on NVIDIA DGX Spark. Get guidance on choosing containers and packages, resolving CUDA compatibility errors, and checking that PyTorch can access the GPU before starting a workload.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up a new DGX Spark for training or inference
• Diagnosing CUDA compatibility and import errors
• Choosing between containers and direct pip installation

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Fine-tune large language models with LoRA and QLoRA while limiting GPU memory use. Guides training settings and adapter management, using small sets of task-specific weights that can be saved, switched, or merged into a base model.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fine-tuning an LLM with limited GPU memory
• Choosing LoRA settings and training layers
• Switching task-specific adapters on one base model

Guides experiments that inspect and change internal model values using nnsight. Covers local PyTorch models and remote experiments through NDIF, including replacing intermediate values to test how particular layers affect a model’s predictions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Inspecting layer outputs and attention patterns
• Replacing internal values to compare predictions
• Running large-model experiments through NDIF

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Deploy AI models on GKE with suitable GPUs or TPUs and model servers. Generate deployment configuration, adjust automatic scaling, and tune inference—the process of producing model responses—for memory use, throughput, and response time.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying Gemma or Llama on GKE
• Choosing GPUs or TPUs for model serving
• Configuring automatic scaling for LLM requests

Write and debug code for further training large language models with TRL. Match training methods to dataset formats, configure supervised or preference-based training, and set up reward functions through Python or the command line.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Training a language model on example responses
• Training with preferred and rejected response pairs
• Debugging GRPO reward functions and batch settings

Fill unfinished research-wiki paper pages using summaries and abstracts from external sources. Organize each paper’s problem, method, results, limitations, and reusable ideas while preserving existing content by default and recording which source supplied the information.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fill paper pages left unfinished after a literature import
• Summarize one paper’s methods, results, and limitations
• Prepare reusable research ideas for later brainstorming

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting article details into JSON
• Building agents that call tools
• Reusing shared prompts across conversations