api
FreeUse liter-llm api reference documentation.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026
FreePublished on GitHub
liter-llm-streaming-responses
Use when streaming tokens incrementally from an LLM via liter-llm over SSE or async iterators. Covers chat_stream, delta handling, and null-content chunks.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263 stars(GitHub 263 + cccskills 0)
Works withOpenAI
Run score Not tested
See how to installFiles included (2)
- SKILL.md2.0 KB
- index.md447 B
SKILL.md (original)
Review the instructions the agent will receive before you install.
<!--
AI-RULEZ :: GENERATED FILE — DO NOT EDIT
Content-Hash: blake3:6de558b3f33f35e9e7ef76dc64d4ebd199a8a831dc28e306698f18fc16933b4e
Source-Hash: blake3:e9a3504d653cf1d11dce92e35c7dc252403c7cac2ca0cae3c0b8c8addd45159e
Schema-Version: v1
-->
Streaming Responses
Use chat_stream(...) to receive tokens as they are produced instead of waiting
for the full completion. The proxy streams over SSE; bindings expose async
iterators.
Python
import asyncio, os
from liter_llm import create_client
from liter_llm._internal_bindings import ChatCompletionRequest
async def main() -> None:
client = create_client(api_key=os.environ["OPENAI_API_KEY"])
request = ChatCompletionRequest.from_json(
'{"model":"openai/gpt-4o","messages":[{"role":"user","content":"Tell me a story"}],"stream":true}'
)
async for chunk in client.chat_stream(request):
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
asyncio.run(main())
TypeScript
import { createClient } from "@xberg-io/liter-llm";
const client = createClient(process.env.OPENAI_API_KEY!);
const chunks = await client.chatStream({
model: "openai/gpt-4o",
messages: [{ role: "user", content: "Tell me a story" }],
});
for await (const chunk of chunks) {
process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
}
Notes
- The first and last chunks often carry null content. Always null-check
chunk.choices[0].delta.content(Python) orchunk.choices[0]?.delta?.content(TypeScript) before using it. - Tool-call deltas arrive in
delta.tool_calls(Python) /delta.toolCalls(TypeScript); accumulatefunction.argumentsfragments across chunks before parsing. - Through the proxy, request streaming with
"stream": trueon/v1/chat/completions.
Reviews
No reviews yet. Tell others what you thought.
Comments
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
More skills from this repository
cli
FreeUse liter-llm cli reference documentation.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026
liter-llm
FreeUniversal LLM API client for 174 providers with native bindings for 14 languages. Use when writing code that calls LLM APIs via liter-llm in Python, TypeScript, Rust, Go, Java, C#, Ruby, PHP, Elixir, WASM, or C, when running the OpenAI-compatible proxy, or when calling LLMs through the MCP server. Covers chat, streaming, tool calling, embeddings, image generation, speech, transcription, moderation, web search, OCR, reranking, provider routing, middleware, and configuration.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026
Use when sending chat completions through liter-llm and routing to a specific provider via the `provider/model` prefix. Covers the chat call shape, provider routing, model_hint, message roles, and error categories.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026
Use when generating embeddings, calling the 12 web-search providers, or running OCR over documents with the 4 OCR providers through liter-llm. Covers embed, search, and ocr methods plus reranking.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026
Use when running the `liter-llm api` OpenAI-compatible gateway — virtual keys, per-key rate limits, budgets, cost tracking, and model routing. Covers the TOML config and the 22 REST endpoints.
The English summary is being prepared. Showing the original description.
xberg-io/liter-llm263Updated Oct 26, 2026