8 skills · sorted by relevance
Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.
Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
The English summary is being prepared. Showing the original description.
Use when working on or testing imp-server and its HTTP APIs - OpenAI/Anthropic endpoints, /v1/chat/completions, /v1/messages, /v1/responses, SSE streaming, tool calling, json_schema/regex/GBNF constrained decoding, thinking/reasoning_content, cache_control/prefix cache, model loading and swapping, request priority, X-Request-Id, OTLP tracing / traceparent / Jaeger, /metrics, container env (IMP_SET), "server returns 404/garbage", API compliance. Do NOT use for CLI-only inference (imp-cli), kernel work (sm120-cuda-expert), or pure output-quality checks (check-degeneration).
The English summary is being prepared. Showing the original description.
Use when needing 100% type-safe, schema-valid data extraction from LLMs. Keywords: structured output, JSON schema, Pydantic, Zod, type-safe, constrained decoding, validation, tool use.
The English summary is being prepared. Showing the original description.
Geração estruturada rápida e serving para LLMs com RadixAttention para cache de prefixos. Use para saídas JSON/regex, decodificação restrita, workflows com agentes que fazem chamadas de ferramentas, ou quando você precisa de 5× mais rápido que vLLM com compartilhamento de prefixos. Alimenta 300.000+ GPUs na xAI, AMD, NVIDIA e LinkedIn.
The English summary is being prepared. Showing the original description.
Get reliable JSON, enums, and typed objects from LLMs using response_format, tool_use, and schema-constrained decoding across OpenAI, Anthropic, and Google APIs.
The English summary is being prepared. Showing the original description.
Get reliable JSON, enums, and typed objects from LLMs using response_format, tool_use, and schema-constrained decoding across OpenAI, Anthropic, and Google APIs.
The English summary is being prepared. Showing the original description.