'serving' 검색 결과

예약한 Itô GPU의 모델 서비스 지원 여부와 향후 확인 사항을 정리합니다. 현재는 서비스를 시작하지 않으며, 미지원 범위와 필요한 조건을 안내합니다.

• Itô GPU 예약 후 서비스 지원을 확인합니다
• OpenAI 호환 접속 주소의 제약을 확인합니다
• 향후 모델 서비스 설정을 검토합니다

[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the endpoint with the standard TTFT/TPOT/goodput protocol. Use when the user says: inference-serving, inference serving, serve this model, serve the model, model serving, serving endpoint, vllm, llama.cpp.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when querying or calling pay-per-token foundation models, or when creating, updating, listing, or querying Unity Gateway (also called Unity AI Gateway) services, including model provider services, MCP services, system.ai services, and three-part Unity Catalog service names (catalog.schema.service). Also use for managing spend budgets and usage alerts on Unity Gateway, and for migrating from legacy AI Gateway. Also use when a workload with Model Serving endpoint permissions (for example CAN_QUERY) gets PERMISSION_DENIED querying a three-part model name, since serving-endpoint access does not grant access to a Unity Gateway service. Not for configuring legacy AI Gateway on Model Serving endpoints; use databricks-model-serving for that.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vLLM으로 대규모 언어 모델을 API로 제공하거나 많은 입력을 일괄 처리하도록 돕습니다. 배포와 성능 모니터링, 메모리 절약, 응답 속도 조정도 다룹니다.

• OpenAI 호환 API를 제공할 때
• 많은 프롬프트를 일괄 처리할 때
• 제한된 GPU 메모리로 모델을 실행할 때

코드의 동작을 유지하며 로직 추출, 중복 통합, 담당 위치 이동을 진행합니다. 구조 변경 전후에 같은 검증을 수행해 동작이 일치하는지 확인합니다.

• 동작을 유지하며 로직을 추출할 때
• 중복된 로직을 통합할 때
• 모듈 사이에서 처리 책임을 옮길 때

중복되거나 검증 내용이 빈약하거나 내부 구현에 의존하는 테스트를 조사합니다. 결함 재발을 감지할 검증을 남기면서 근거에 따라 테스트를 정리하거나 삭제합니다.

• 중복 테스트 감사
• 회귀 감지를 유지하며 테스트 정리
• 내부 구현에 의존하는 테스트 검토

Build Python workloads on Modal for cloud execution, including GPU model training and inference, web APIs, scheduled jobs, and parallel batch processing. Configure container environments, persistent storage, and automatic scaling through the Modal SDK.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Serving an AI model through a web API
• Running model training or fine-tuning on cloud GPUs
• Processing independent batch inputs in parallel

Polish or shorten existing academic prose, or translate Chinese drafts into English, while preserving facts, terminology, and the limits of the evidence. Adapt revisions to manuscript sections and journal context, and address LaTeX layout problems.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Polishing an existing English academic manuscript
• Translating a Chinese academic draft into English
• Shortening Results or main text while preserving supporting evidence

Calculate how to divide serving memory between recurrent state and KV cache, two memory pools used by hybrid models. Use startup logs and workload settings to recommend a ratio, estimate concurrency, and identify the limiting pool.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choosing a memory ratio for a hybrid model
• Investigating limits on concurrent requests
• Recalculating memory allocation for context length

Investigate SGLang serving failures by collecting server state, preserving failing requests, and replaying them on a clean target. Summarize evidence and likely causes, then choose the next debugging tool for crashes, stalls, or performance regressions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate failed health checks and growing queues
• Reproduce request-specific timeouts or crashes
• Investigate output changes after deployment

Create disk images for investigations using dd or dcfldd on Linux. The workflow covers write protection, bit-for-bit copying, hash checks to verify integrity, and documentation of the acquisition process.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preserving a suspect drive before investigation
• Creating evidence copies of USB devices or memory cards
• Verifying that a disk image matches its source

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Add GBrain memory to an existing agent or connect a hosted brain while preserving current instructions and identity. Start with storage and keyword search that need no API key, then verify saving, retrieval, and recall across conversations.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Add memory to an existing Claude Code or Codex agent
• Connect an agent to a hosted brain
• Diagnose an incomplete installation while preserving memory

Move a shared NanoClaw Slack bot to separate apps for existing agent groups while preserving their workspaces, memory, and response settings. The skill guides a resumable migration or records the choice to keep the supported classic setup.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Giving existing agents separate Slack bot identities
• Preserving response settings during migration
• Resuming an interrupted migration

Reorganize NemoClaw documentation into focused pages with clearer navigation and a single home for each topic. Plan or implement structural changes while preserving useful information and existing published URLs, then validate routes, links, and builds.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Splitting long documentation pages into focused topics
• Consolidating duplicated troubleshooting and reference content
• Reorganizing navigation while preserving published URLs

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Guides adapting OpenPI robot models to custom datasets and serving them to produce actions from observations. Covers JAX and PyTorch training, checkpoint conversion, and troubleshooting for normalization statistics and GPU memory.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fine-tune OpenPI models on custom robot datasets
• Convert JAX checkpoints to PyTorch
• Serve policies for ALOHA, DROID, or LIBERO

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting article details into JSON
• Building agents that call tools
• Reusing shared prompts across conversations

Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose a container for an LLM deployment
• Select images for embeddings and rerankers
• Check regional image addresses and GPU compatibility

Plan how to serve a model on Amazon SageMaker AI. The skill uses model type, traffic, response time, and cost constraints to recommend a deployment route, check instance quotas, and coordinate specialized setup and deployment skills.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Plan AWS hosting for a language model
• Choose a serving route for an embedding model
• Plan asynchronous inference for image generation

Create and edit Google Docs while preserving requested structure, formatting, tabs, and tables. Adapt templates using current source information, update links and smart chips, and check that the finished document covers the requested content.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Create a document from a Google Docs template
• Update text and tables while preserving formatting
• Adapt a past project document for new work