“serving”的搜索结果

针对在Itô已预订GPU上进行模型服务的咨询，说明当前尚未支持的范围，并整理未来所需的配置、资格验证及确认事项；目前不会启动模型服务。

• 确认预订Itô GPU后的服务支持情况
• 了解OpenAI兼容端点的限制
• 查看未来模型服务所需配置

[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the endpoint with the standard TTFT/TPOT/goodput protocol. Use when the user says: inference-serving, inference serving, serve this model, serve the model, model serving, serving endpoint, vllm, llama.cpp.

中文简介正在准备中。当前显示原文说明。

Provides instructions for serving language and vision-language models on Jetson with vLLM or SGLang. Selects a runtime for the device and JetPack version, supplies launch commands, and explains how to verify the OpenAI-compatible API.

中文简介正在准备中。当前显示英语简介。

• Starting a language-model API on Jetson
• Connecting Open WebUI or an app to a model server
• Serving a vision-language model with vLLM

Use when querying or calling pay-per-token foundation models, or when creating, updating, listing, or querying Unity Gateway (also called Unity AI Gateway) services, including model provider services, MCP services, system.ai services, and three-part Unity Catalog service names (catalog.schema.service). Also use for managing spend budgets and usage alerts on Unity Gateway, and for migrating from legacy AI Gateway. Also use when a workload with Model Serving endpoint permissions (for example CAN_QUERY) gets PERMISSION_DENIED querying a three-part model name, since serving-endpoint access does not grant access to a Unity Gateway service. Not for configuring legacy AI Gateway on Model Serving endpoints; use databricks-model-serving for that.

中文简介正在准备中。当前显示原文说明。

协助配置vLLM，将大语言模型作为API提供服务或批量生成文本，涵盖部署、性能监控、量化和多GPU分工，也用于排查显存不足与响应缓慢。

• 提供兼容OpenAI的API
• 批量处理大量提示词
• 在有限显存下运行模型

在保持代码行为的前提下，提取逻辑、合并重复内容或转移职责。结构调整前后使用相同方法验证，确认行为一致，并将结构整理与新功能分开处理。

• 保持行为不变地提取逻辑
• 合并重复逻辑
• 在模块之间转移职责

检查重复、缺乏实质验证或依赖内部实现的测试，逐项确认删除依据与保留的替代测试，在保留问题复发检测能力的前提下，合并、修复或删除测试。

• 审查重复测试
• 保留回归检测能力并整理测试
• 重新评估依赖内部实现的测试

Build Python workloads on Modal for cloud execution, including GPU model training and inference, web APIs, scheduled jobs, and parallel batch processing. Configure container environments, persistent storage, and automatic scaling through the Modal SDK.

中文简介正在准备中。当前显示英语简介。

• Serving an AI model through a web API
• Running model training or fine-tuning on cloud GPUs
• Processing independent batch inputs in parallel

Polish or shorten existing academic prose, or translate Chinese drafts into English, while preserving facts, terminology, and the limits of the evidence. Adapt revisions to manuscript sections and journal context, and address LaTeX layout problems.

中文简介正在准备中。当前显示英语简介。

• Polishing an existing English academic manuscript
• Translating a Chinese academic draft into English
• Shortening Results or main text while preserving supporting evidence

Calculate how to divide serving memory between recurrent state and KV cache, two memory pools used by hybrid models. Use startup logs and workload settings to recommend a ratio, estimate concurrency, and identify the limiting pool.

中文简介正在准备中。当前显示英语简介。

• Choosing a memory ratio for a hybrid model
• Investigating limits on concurrent requests
• Recalculating memory allocation for context length

Investigate SGLang serving failures by collecting server state, preserving failing requests, and replaying them on a clean target. Summarize evidence and likely causes, then choose the next debugging tool for crashes, stalls, or performance regressions.

中文简介正在准备中。当前显示英语简介。

• Investigate failed health checks and growing queues
• Reproduce request-specific timeouts or crashes
• Investigate output changes after deployment

Create disk images for investigations using dd or dcfldd on Linux. The workflow covers write protection, bit-for-bit copying, hash checks to verify integrity, and documentation of the acquisition process.

中文简介正在准备中。当前显示英语简介。

• Preserving a suspect drive before investigation
• Creating evidence copies of USB devices or memory cards
• Verifying that a disk image matches its source

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

中文简介正在准备中。当前显示英语简介。

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

中文简介正在准备中。当前显示英语简介。

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.

中文简介正在准备中。当前显示英语简介。

• Deploy an OpenAI-compatible language model API
• Generate text from many prompts in a batch
• Serve quantized models with limited GPU memory

Add GBrain memory to an existing agent or connect a hosted brain while preserving current instructions and identity. Start with storage and keyword search that need no API key, then verify saving, retrieval, and recall across conversations.

中文简介正在准备中。当前显示英语简介。

• Add memory to an existing Claude Code or Codex agent
• Connect an agent to a hosted brain
• Diagnose an incomplete installation while preserving memory

Move a shared NanoClaw Slack bot to separate apps for existing agent groups while preserving their workspaces, memory, and response settings. The skill guides a resumable migration or records the choice to keep the supported classic setup.

中文简介正在准备中。当前显示英语简介。

• Giving existing agents separate Slack bot identities
• Preserving response settings during migration
• Resuming an interrupted migration

Reorganize NemoClaw documentation into focused pages with clearer navigation and a single home for each topic. Plan or implement structural changes while preserving useful information and existing published URLs, then validate routes, links, and builds.

中文简介正在准备中。当前显示英语简介。

• Splitting long documentation pages into focused topics
• Consolidating duplicated troubleshooting and reference content
• Reorganizing navigation while preserving published URLs

Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.

中文简介正在准备中。当前显示英语简介。

• Extracting article details into JSON
• Building agents that call tools
• Reusing shared prompts across conversations

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

中文简介正在准备中。当前显示英语简介。

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

中文简介正在准备中。当前显示英语简介。

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Guides adapting OpenPI robot models to custom datasets and serving them to produce actions from observations. Covers JAX and PyTorch training, checkpoint conversion, and troubleshooting for normalization statistics and GPU memory.

中文简介正在准备中。当前显示英语简介。

• Fine-tune OpenPI models on custom robot datasets
• Convert JAX checkpoints to PyTorch
• Serve policies for ALOHA, DROID, or LIBERO

Plan how to serve a model on Amazon SageMaker AI. The skill uses model type, traffic, response time, and cost constraints to recommend a deployment route, check instance quotas, and coordinate specialized setup and deployment skills.

中文简介正在准备中。当前显示英语简介。

• Plan AWS hosting for a language model
• Choose a serving route for an embedding model
• Plan asynchronous inference for image generation

Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.

中文简介正在准备中。当前显示英语简介。

• Choose a container for an LLM deployment
• Select images for embeddings and rerankers
• Check regional image addresses and GPU compatibility