1.3K skills · sorted by relevance
Clarify the availability and requirements of model serving on booked Itô GPUs. The supplied implementation does not support serving, so it reports that limitation and documents future booking verification, configuration review, and explicit confirmation requirements.
[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the endpoint with the standard TTFT/TPOT/goodput protocol. Use when the user says: inference-serving, inference serving, serve this model, serve the model, model serving, serving endpoint, vllm, llama.cpp.
The English summary is being prepared. Showing the original description.
Use when querying or calling pay-per-token foundation models, or when creating, updating, listing, or querying Unity Gateway (also called Unity AI Gateway) services, including model provider services, MCP services, system.ai services, and three-part Unity Catalog service names (catalog.schema.service). Also use for managing spend budgets and usage alerts on Unity Gateway, and for migrating from legacy AI Gateway. Also use when a workload with Model Serving endpoint permissions (for example CAN_QUERY) gets PERMISSION_DENIED querying a three-part model name, since serving-endpoint access does not grant access to a Unity Gateway service. Not for configuring legacy AI Gateway on Model Serving endpoints; use databricks-model-serving for that.
The English summary is being prepared. Showing the original description.
Set up vLLM to serve large language models through an OpenAI-compatible API or process prompts in batches. The skill covers deployment, performance monitoring, and quantization, which reduces model memory requirements.
Restructure code through extraction, consolidation, or responsibility moves while preserving its behavior. Establish checks before editing, repeat them afterward, and keep intermediate versions buildable and testable without mixing in feature changes.
Audit a test suite to identify redundant tests, empty checks, and tests tied to internal implementation. Consolidate or remove them with evidence while preserving useful checks that detect regressions, meaning previously fixed problems returning.
Build Python workloads on Modal for cloud execution, including GPU model training and inference, web APIs, scheduled jobs, and parallel batch processing. Configure container environments, persistent storage, and automatic scaling through the Modal SDK.
Polish or shorten existing academic prose, or translate Chinese drafts into English, while preserving facts, terminology, and the limits of the evidence. Adapt revisions to manuscript sections and journal context, and address LaTeX layout problems.
Calculate how to divide serving memory between recurrent state and KV cache, two memory pools used by hybrid models. Use startup logs and workload settings to recommend a ratio, estimate concurrency, and identify the limiting pool.
Investigate SGLang serving failures by collecting server state, preserving failing requests, and replaying them on a clean target. Summarize evidence and likely causes, then choose the next debugging tool for crashes, stalls, or performance regressions.
Create disk images for investigations using dd or dcfldd on Linux. The workflow covers write protection, bit-for-bit copying, hash checks to verify integrity, and documentation of the acquisition process.
Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Deploy language models with vLLM through an OpenAI-compatible API or offline batch processing. Tune response speed, throughput, and GPU memory use with quantization, which reduces numerical precision, and execution across multiple GPUs.
Add GBrain memory to an existing agent or connect a hosted brain while preserving current instructions and identity. Start with storage and keyword search that need no API key, then verify saving, retrieval, and recall across conversations.
Move a shared NanoClaw Slack bot to separate apps for existing agent groups while preserving their workspaces, memory, and response settings. The skill guides a resumable migration or records the choice to keep the supported classic setup.
Reorganize NemoClaw documentation into focused pages with clearer navigation and a single home for each topic. Plan or implement structural changes while preserving useful information and existing published URLs, then validate routes, links, and builds.
Guides adapting OpenPI robot models to custom datasets and serving them to produce actions from observations. Covers JAX and PyTorch training, checkpoint conversion, and troubleshooting for normalization statistics and GPU memory.
Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.
Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.
Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.
Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.
Plan how to serve a model on Amazon SageMaker AI. The skill uses model type, traffic, response time, and cost constraints to recommend a deployment route, check instance quotas, and coordinate specialized setup and deployment skills.
Create and edit Google Docs while preserving requested structure, formatting, tabs, and tables. Adapt templates using current source information, update links and smart chips, and check that the finished document covers the requested content.