The Ollama REST API

The daemon is a plain HTTP server on the loopback interface. Everything below is the shape this install answers with - not a client library's wrapper.

The address

Ollama binds 127.0.0.1:11434 unless OLLAMA_HOST says otherwise. The value is host:port (a scheme is accepted). Two different readers use it:

• the daemon reads OLLAMA_HOST as the address to BIND (0.0.0.0:11434 to serve the whole network);
• a client reads it as the address to CONNECT to.

That asymmetry is the classic mistake: setting OLLAMA_HOST=0.0.0.0:11434 for the server and then pointing a client at http://0.0.0.0:11434 works on Linux and fails on Windows. Clients should default to http://127.0.0.1:11434.

The first request of any session should be the version, because every other behaviour is version-dependent:

curl -s http://127.0.0.1:11434/api/version

{"version":"0.40.3"}

dsh-ollama's ollama_status answers this and the model inventory in one call; when you are writing a program, this is the endpoint to probe.

The three request shapes

One prompt: /api/generate

curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:7b","prompt":"Why is the sky blue?","stream":false}'

stream: false answers ONE JSON object carrying the whole completion and every timing field. stream: true (the daemon's default) answers NDJSON - see below.

A conversation: /api/chat

curl -s http://127.0.0.1:11434/api/chat -d '{"model":"qwen2.5:7b","messages":[{"role":"system","content":"Be brief."},{"role":"user","content":"Why is the sky blue?"}],"stream":false}'

Roles are system, user, assistant and tool. The same model, template and options apply; the difference is that the daemon applies the chat template to the whole array instead of wrapping one prompt.

Vectors: /api/embed

curl -s http://127.0.0.1:11434/api/embed -d '{"model":"nomic-embed-text","input":["the cat sat","a cat was sitting"]}'

input is a string or an array. The answer is {"embeddings": [[...], [...]], "model": "...", ...} - one vector per input, in order. Older daemons only have the single-prompt form:

curl -s http://127.0.0.1:11434/api/embeddings -d '{"model":"nomic-embed-text","prompt":"the cat sat"}'

That answers {"embedding": [...]}. Prefer /api/embed: one request, many inputs, and a dimension you can rely on. If /api/embed returns 404, fall back.

Streaming is NDJSON

A streaming answer is one JSON object per line, each line terminated by \n, with done: false on every partial object and done: true on the last one. The partial objects carry response (generate) or message.content (chat); the FINAL object carries the token counts and the durations. Nothing before the last line has a measurement in it.

curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:7b","prompt":"Say hello"}'

Three properties follow from that shape, and each one is a bug waiting to happen:

1. A line is not a document. Parse per line; do not buffer the whole body and JSON.parse it.
2. The last line is the measurement. If your reader stops at the first done: true it has the numbers; if it stops at end-of-stream it may have them too, but the done object is the contract.
3. A stream can be cut. A cancelled request, a killed daemon or a timeout ends the stream with no done object at all. Treat "no done object" as a failed measurement, not as an empty answer.

A reader that keeps the text and the final object:

const response = await fetch('http://127.0.0.1:11434/api/generate', {
  method: 'POST',
  headers: { 'content-type': 'application/json' },
  body: JSON.stringify({ model: 'qwen2.5:7b', prompt: 'Say hello', stream: true }),
})
if (!response.ok) throw new Error(`HTTP ${response.status}: ${await response.text()}`)
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
let text = ''
let final = null
for (;;) {
  const { value, done } = await reader.read()
  if (done) break
  buffer += decoder.decode(value, { stream: true })
  let newline = buffer.indexOf('\n')
  while (newline >= 0) {
    const line = buffer.slice(0, newline).trim()
    buffer = buffer.slice(newline + 1)
    newline = buffer.indexOf('\n')
    if (line === '') continue
    const item = JSON.parse(line)
    if (item.response) text += item.response
    if (item.done === true) final = item
  }
}
if (final === null) throw new Error('the stream ended without a done object')
console.log(text, final.eval_count, final.eval_duration)

(The arithmetic that turns those two numbers into tok/s is the ollama-inference-metrics skill's subject.)

The final object

/api/generate with stream: false and the last line of a streaming call have the same fields:

field	unit	what it covers
response	string	the generated text (chat: message.content)
done	bool	true on the final object
done_reason	string	stop (the model stopped) or length (it hit num_predict/context)
context	int[]	the token ids of prompt+answer - pass it back as context to continue
total_duration	ns	the whole daemon-side request
load_duration	ns	loading the weights (0 when they were resident)
prompt_eval_count	int	prompt tokens evaluated
prompt_eval_duration	ns	evaluating the prompt
eval_count	int	tokens generated
eval_duration	ns	generating them

done_reason: "length" is the one field a reader must not ignore: the answer was CUT by your num_predict or the context window, not finished by the model.

Options and keep_alive

options is the sampling/runtime map. The ones that change the measurement:

option	range	effect
num_predict	-1..N	max tokens to generate; -1 (the default) means "until the model stops or the context fills"
temperature	0..2	0 = greedy; the only defensible setting for a reproducible benchmark
seed	int	fixes sampling when temperature > 0; set it anyway so a rerun is comparable
num_ctx	128..	context window in tokens; the single biggest memory/speed knob
top_k, top_p, min_p		sampling cutoffs
repeat_penalty, repeat_last_n		repetition control
stop	string[]	stop sequences
num_gpu	int	layers offloaded to the GPU (0 = CPU only)
num_thread	int	CPU threads
use_mmap, use_mlock	bool	weight paging

Anything the daemon does not recognise in options is silently ignored - which is why a typo is worse than an error: the run reports numbers taken at a setting nobody chose. dsh-ollama closes the map for exactly that reason.

keep_alive is a TOP-LEVEL sibling of options, not an option:

• "5m", "30s", "1h" - keep the weights loaded that long after the request;
• 0 - unload as soon as this request finishes;
• -1 - never unload.

curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:7b","prompt":"hi","keep_alive":"10m"}'

Unloading is the documented use of an empty request:

curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:7b","keep_alive":0}'

That is the call that makes a "cold" benchmark honest: without it, run 1 is as warm as run 2 and the load time you meant to measure never appears.

Models: list, inspect, pull, delete

curl -s http://127.0.0.1:11434/api/tags

/api/tags is the inventory: name, size (bytes), digest (sha256:...), modified_at, and details (family, families, parameter_size, quantization_level, format).

curl -s http://127.0.0.1:11434/api/ps

/api/ps is what is LOADED right now: size, size_vram, context_length, expires_at, and the same details. size_vram is 0 for a CPU-resident model; size_vram / size is the offload share.

curl -s http://127.0.0.1:11434/api/show -d '{"model":"qwen2.5:7b"}'

/api/show is the model's own metadata: details, parameters (the Modelfile defaults - read them before assuming temperature), template, license, model_info (general.architecture, *.context_length, *.embedding_length, *.block_count) and capabilities (completion, tools, embedding, vision, thinking depending on the model). capabilities is what tells you whether a tag can embed at all.

curl -s http://127.0.0.1:11434/api/pull -d '{"model":"qwen2.5:7b","stream":true}'

A pull is NDJSON progress, not a completion: {"status":"pulling manifest"}, then one line per layer with digest, total and completed, then {"status":"verifying sha256 digest"}, then {"status":"success"}. It is measured in gigabytes and minutes; there is no partial-success state - a cut pull leaves layers cached and the next pull resumes them.

curl -s -X DELETE http://127.0.0.1:11434/api/delete -d '{"model":"llama3.2:1b"}'

Delete answers 200 with an empty body. It frees the bytes immediately; the model comes back only by pulling it again. dsh-ollama requires the tag repeated as confirm for this reason.

The other model verbs, for completeness:

curl -s http://127.0.0.1:11434/api/copy -d '{"source":"qwen2.5:7b","destination":"qwen2.5:7b-mine"}'
curl -s http://127.0.0.1:11434/api/create -d '{"model":"my-model","from":"qwen2.5:7b","system":"You answer in one sentence."}'

copy and create are not tools in this pack - they mutate the model store in ways an agent should not do implicitly - but they are part of the surface and worth knowing when a user asks why a tag exists.

Errors, and what they mean

Ollama reports a problem as a non-2xx status with a JSON body {"error": "..."}.

status	body	meaning	the fix
404	model "x" not found, try pulling it first	the tag is not on disk	ollama pull x
404	404 page not found	the route does not exist on this daemon (an older build, or a typo)	check /api/version, use the older shape
400	invalid options / json: cannot unmarshal ...	the body's shape is wrong	compare with this page
500	...	the model failed while loading or running	read the message; often memory (num_ctx too large)
(none)	fetch failed / ECONNREFUSED	the DAEMON is not running	ollama serve, or the desktop app

A 500 whose message mentions memory is not a bug in your code: num_ctx sizes the KV cache, and a context the machine cannot hold fails at load. Try a smaller num_ctx or a smaller quantization.

Structured output

format is a top-level field. "json" asks for any valid JSON; a JSON Schema asks for that shape and is the reliable form:

curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:7b","prompt":"Give me one user: name and age.","format":{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]},"stream":false}'

The answer's text is a JSON document (not a Markdown fence). Parse it and fail loudly when it does not parse - a schema is a strong request, not a guarantee.

Tool calling

/api/chat accepts tools (a JSON-Schema function list) in the same shape as the OpenAI tools array. The model answers with message.tool_calls: [{ function: { name, arguments } }] and no content; you run the function, append a {role: "tool", content: "...", tool_name: "..."} message, and call again.

curl -s http://127.0.0.1:11434/api/chat -d '{"model":"qwen2.5:7b","messages":[{"role":"user","content":"What is the weather in Lisbon?"}],"tools":[{"type":"function","function":{"name":"weather","description":"Current weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"stream":false}'

Not every model supports tools: check capabilities from /api/show first, or you will get a plain-text answer that looks like a refusal.

The OpenAI-compatible endpoints

Ollama also serves /v1: the same daemon, the OpenAI wire format. Use it when a client library or SDK already speaks OpenAI and you cannot change it.

curl -s http://127.0.0.1:11434/v1/chat/completions -d '{"model":"qwen2.5:7b","messages":[{"role":"user","content":"hi"}]}'
curl -s http://127.0.0.1:11434/v1/completions -d '{"model":"qwen2.5:7b","prompt":"hi"}'
curl -s http://127.0.0.1:11434/v1/embeddings -d '{"model":"nomic-embed-text","input":"hi"}'
curl -s http://127.0.0.1:11434/v1/models

Two cost-saving facts about this surface:

• the timings are not in the OpenAI body; the usage object carries token counts only. If you need the durations, use the native /api route;
• stream: true here is Server-Sent Events (data: {...} lines ending data: [DONE]), not NDJSON. A reader written for /api will not parse it.

The CLI, for the same operations

task	API	CLI
list models	GET /api/tags	ollama list
loaded models	GET /api/ps	ollama ps
inspect	POST /api/show	ollama show qwen2.5:7b
pull	POST /api/pull	ollama pull qwen2.5:7b
delete	DELETE /api/delete	ollama rm qwen2.5:7b
one prompt	POST /api/generate	ollama run qwen2.5:7b "hi"
daemon	-	ollama serve

ollama run starts an interactive session when given no prompt; it is not a drop-in for a scripted call, and it prints the finished text without the timing fields.

Environment variables that change the daemon

These are read by the SERVER at start, not per request:

variable	default	what it does
OLLAMA_HOST	127.0.0.1:11434	the bind address
OLLAMA_MODELS	~/.ollama/models	where the weights live
OLLAMA_KEEP_ALIVE	5m	default residency for a loaded model
OLLAMA_NUM_PARALLEL	1 (auto)	concurrent requests one model serves; each gets its own KV cache slot
OLLAMA_MAX_LOADED_MODELS	3 x GPUs (auto)	how many models may be resident at once
OLLAMA_MAX_QUEUE	512	queued requests before the daemon answers 503
OLLAMA_FLASH_ATTENTION	off	a speed/memory trade at long context
OLLAMA_KV_CACHE_TYPE	f16	q8_0 / q4_0 shrink the KV cache
OLLAMA_ORIGINS	localhost only	CORS origins the daemon accepts from a browser
OLLAMA_DEBUG	off	verbose server logging

Two settings interact in a way that ruins benchmarks: with OLLAMA_NUM_PARALLEL=2, one model serves two requests at once, so two "sequential" runs can overlap and both report a lower tok/s. Measure with parallelism at its default, or state it in the result.

What not to do

• Do not assume the daemon is running. Probe /api/version and say the address you tried.
• Do not buffer a streaming response and parse it as one JSON document.
• Do not invent an options key: an unknown one is ignored, and the numbers become a claim about a setting the daemon never applied.
• Do not quote total_duration as latency: it is the daemon's own span, and it excludes the transport.
• Do not treat a 404 for a model as "the daemon is broken": the fix is ollama pull, and the error body says so.