Chat and models API
Base URL and authentication are on the REST API overview.
Models
GET /v1/models
OpenAI-compatible model list.
Response 200
{
"object": "list",
"data": [
{ "id": "llama3.2:1b", "object": "model", "owned_by": "dppan" }
]
}GET /api/models
Live node coverage plus multimodal capability metadata for models currently represented by registered nodes.
Response 200
[
{
"id": "gemma4:12b",
"n_layers": 48,
"d_model": 3840,
"node_count": 2,
"active_nodes": 2,
"idle_nodes": 0,
"dead_nodes": 0,
"covered": [[0, 24], [24, 48]],
"gaps": [],
"chain_complete": true,
"total_vram_mb": 32768,
"input_modalities": ["text", "image", "audio", "video_frames"],
"supports_tools": true,
"reasoning_effort_levels": [],
"projector_state": "ready",
"decoder_requirements": {
"m_rope": false,
"non_causal_media": true
},
"unsupported_reason": null
}
]reasoning_effort_levels lists the reasoning_effort values this model accepts. It is derived when the model loads, not tabulated, and comes from one of two mechanisms:
- Native — the model's chat template reads an effort variable, so the level is passed to it and the model was trained on the distinction. Probed by rendering, because support does not follow architecture (Qwen3.6 has none, Qwen3.8 does) and the variable is not always spelled the same (
gpt-ossand Qwen3.8 readreasoning_effort; Muse-Glimmer readsreasoning_strength). Levels are whatever that model accepts. - Token budget — the model reasons but its template exposes no control at all (Kimi-K2-Thinking, DeepSeek-R1). The level becomes a cap on how many tokens the reasoning span may use, after which its closing tag is forced. Levels are always
["low", "medium", "high", "max"]— 512, 2048, 8192 tokens and no limit. Requires the template to declare reasoning tags to close.
Native wins when a model has both, so the list never mixes the two. A model with neither returns [] and has no effort control; render no selector for those rather than a disabled one.
"none" is accepted by the chat endpoints on every model and is deliberately absent from this list — it is applied as "reasoning off" and never reaches a template, so probing it would report it unsupported on the models where it is the one value that always works.
GET /models/registry
Authoritative catalog plus local config/models.toml overlay. Unlike GET /api/models, this includes models with no currently registered nodes.
Response 200
[
{
"model_id": "gemma4:12b",
"n_layers": 48,
"d_model": 3840,
"gguf_sha256": "",
"quant_type": "Q4_0",
"arch": "gemma4",
"engine_loaded": false,
"source": "catalog",
"input_modalities": ["text", "image", "audio", "video_frames"],
"projector_state": "ready",
"decoder_requirements": {
"m_rope": null,
"non_causal_media": null
},
"unsupported_reason": null
},
{
"model_id": "llama3.2:1b",
"n_layers": 16,
"d_model": 2048,
"gguf_sha256": "74701a8c...",
"quant_type": "Q4_K_M",
"arch": "llama",
"engine_loaded": true,
"source": "local",
"input_modalities": ["text"],
"projector_state": "missing",
"decoder_requirements": {
"m_rope": null,
"non_causal_media": null
},
"unsupported_reason": null
}
]projector_state is ready, missing, or invalid. A valid vision projector also advertises video_frames. Decoder requirement values are null until the projector first loads, then true or false. unsupported_reason says why a projector found on disk cannot be used.
GET /admin/chain/:model_id (admin)
Chain health for a specific model — shows which layer ranges are covered.
Response 200
{
"model_id": "llama3.2:1b",
"n_layers": 16,
"covered": [[0, 8], [8, 16]],
"gaps": [],
"ready": true
}Chat Completions
POST /v1/chat/completions
OpenAI-compatible chat endpoint (non-streaming and SSE streaming).
Request body
{
"model": "llama3.2:1b",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "What is 2 + 2?" }
],
"max_tokens": 200,
"temperature": 0.8,
"top_p": 0.95,
"stream": false
}| Field | Type | Default | Description |
|---|---|---|---|
model | string | required | Model ID from the registry |
messages | array | required | Conversation; role is system, user, or assistant |
max_tokens | integer | 512 | Max new tokens to generate |
temperature | float | 0.8 | Sampling temperature (0 = greedy) |
top_p | float | 0.95 | Nucleus sampling p |
thinking | boolean | model default | Force reasoning on or off |
reasoning_effort | string | model default | How hard to think — see below |
stream | boolean | false | Enable SSE streaming |
reasoning_effort
OpenAI's spelling. "none" turns reasoning off and works on every model. Any other value must be one of the model's reasoning_effort_levels from GET /api/models; anything else is a 400 listing what the model accepts.
How a level is applied depends on the model, and the two mechanisms are described under GET /api/models above. On a native model the level reaches the model's own chat template, so it means whatever that model was trained to make it mean. On a token-budget model it caps the reasoning span instead: low 512 tokens, medium 2048, high 8192, max no limit. When the cap is reached the model's own reasoning end tag is forced, so the reply still parses normally — it just stops deliberating and answers.
The rejection is deliberate. A template that does not recognise a value may raise rather than ignore it — Qwen3.8 raises on OpenAI's own minimal and max — and a raised template silently falls back to a plainer prompt with none of the model's reasoning instructions. Failing the request is the only way that surfaces.
A request that omits the field renders exactly as it did before the field existed. thinking and reasoning_effort are independent: the first decides whether the model reasons, the second how hard, and "none" sets the first.
The same field is accepted on the /ws/chat request frame. POST /v1/responses does not accept it yet.
messages[].content may also be an ordered array of multimodal parts. Plain string content remains fully backward-compatible.
{
"model": "gemma4:12b",
"messages": [{
"role": "user",
"content": [
{ "type": "text", "text": "Compare these frames." },
{
"type": "image_url",
"image_url": { "url": "data:image/png;base64,<base64>" }
},
{
"type": "input_audio",
"input_audio": { "data": "<base64>", "format": "wav" }
},
{
"type": "input_video",
"input_video": {
"frames": [
{
"image_url": { "url": "data:image/jpeg;base64,<base64>" },
"timestamp_ms": 0
},
{
"image_url": { "url": "data:image/jpeg;base64,<base64>" },
"timestamp_ms": 1000
}
]
}
}
]
}]
}Video: frames or a whole file
input_video takes either frames or file, never both. The example above sends frames the client extracted itself. To hand the server a whole video and let it decode the frames:
{
"type": "input_video",
"input_video": { "file": "data:video/mp4;base64,<base64>" }
}file is the data URL itself, not an object. Accepted containers are video/mp4, video/quicktime, video/webm, and video/x-matroska; the declared MIME type must match the container's magic bytes.
Server-side decoding requires ffmpeg and ffprobe on the host's PATH. Deployments without them report supports_video_files: false on /api/models and reject file with an explanatory error — check that field before uploading. Frame lists work regardless. The server decodes at most 240 frames from one file.
Generation from a chat turn
A model can produce audio mid-conversation by calling a generation tool. Those tools are opt-in: they are offered only when the request asks for them with tool_choice ("auto", "required", or a named function). Without it an ordinary chat turn is unchanged, and cannot hand a probabilistic model the authority to start expensive work nobody requested.
When a reply starts one, the response carries a generated array of job references alongside the usual content, and the audio is fetched the same way as any other generation.
Three optional fields steer what the model produces. Each is the caller's choice rather than the model's — it has no way to know a user's budget or which voice they picked in a UI:
| Field | |
|---|---|
generation_model | Restrict generation to one model. Naming an unconfigured one offers no tools rather than falling back. |
generation_voice | Restrict it to one voice, likewise. |
generation_max_duration_seconds | Ceiling on the audio a reply may produce. |
A reply may start at most four generations; anything beyond that is handed back as an ordinary tool call rather than silently dropped.
Limits
Supported image MIME types are PNG, JPEG, WebP, and BMP. WebP is decoded and re-encoded as PNG before it reaches the model, so its byte budget is counted against the PNG, which is usually larger than the WebP you sent. Audio format is wav or mp3. Video frames must be in nondecreasing timestamp order. Remote HTTP URLs and local paths are disabled; images, video frames, and video files must use base64 data URLs. Requests are limited to 24 MiB, 64 content parts, 16 media items, 16 frames per video part, 8 MiB encoded per item, 16 MiB encoded total, 16 megapixels per image, 32 megapixels total, and five minutes per audio item. HTTP streaming, HTTP non-streaming, and /ws/chat use the same preparation and validation path.
Response 200 (non-streaming)
{
"id": "chatcmpl-<uuid>",
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "4" },
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 1,
"total_tokens": 13
}
}When the model reasoned, its thinking comes back beside the answer rather than inside it:
"message": {
"role": "assistant",
"reasoning_content": "The user wants 2+2. That is 4.",
"content": "4"
}reasoning_content is absent when the model did not reason, so a client that ignores unknown fields always finds the answer alone in content. The name is the one vLLM, SGLang and DeepSeek use; OpenAI's chat schema has no equivalent field, and reasoning tokens are counted in completion_tokens like any other.
Reasoning is spent from the same max_tokens budget as the answer. A model that thinks for longer than the budget allows returns its reasoning with an empty content — raise MAX_TOKENS (default 512) or lower the effort level. finish_reason is "length" when that happens, which is how a client tells it apart from a model that simply had little to say.
finish_reason takes three values, on both the non-streaming response and the final SSE frame:
| Value | Meaning |
|---|---|
stop | The model finished on its own or hit a stop sequence |
length | max_tokens was reached; the reply is cut off |
tool_calls | The reply is a tool call for the caller to run |
length outranks tool_calls: a reply stopped at the cap may have been cut mid-call, so the truncation is reported instead.
Response 200 (streaming, Content-Type: text/event-stream)
Each event:
data: {"choices":[{"delta":{"content":"4"},"finish_reason":null}]}
data: [DONE]Errors
| Code | Meaning |
|---|---|
503 | No Ready nodes covering all layers of the requested model (chain gap) |
422 | Model not in registry |
400 | Malformed request |
Media validation and capability failures return a stable JSON envelope:
{
"error": {
"message": "unsupported modality: ...",
"type": "unsupported_modality",
"param": null,
"code": "unsupported_modality"
}
}DELETE /sessions/:id
Close a chat session and free its memory on every node.
Response 204 — no body
Files (OpenAI-compatible)
An adapter over Artifacts, not a second store. A file is an artifact: same IDs, same ownership, same quota, same retention — so a clip uploaded here can be referenced from DPPAN-native endpoints, and the reverse. Point an OpenAI client at this base URL and files.create() works.
| Method | Path | |
|---|---|---|
POST | /v1/files | Multipart upload. Fields: purpose, file. Returns 201. |
GET | /v1/files | List your own files. |
GET | /v1/files/:id | Metadata. |
GET | /v1/files/:id/content | The bytes. |
DELETE | /v1/files/:id | Delete. |
curl -X POST http://localhost:8001/v1/files \
-F purpose=user_data -F [email protected]
# {"id":"66815206-...","object":"file","bytes":240044,
# "filename":"my-voice.wav","purpose":"user_data"}The returned id is an artifact ID, so it can be passed straight to a generation as speaker_artifact.
Where compatibility stops
purpose is validated rather than accepted. assistants, user_data and vision work; fine-tune, batch and evals describe workflows this deployment does not run and are refused by name rather than stored — accepting one would report success for something that will never happen.
Uploads are capped at 64 MiB. Multipart arrives whole rather than streaming, so the memory cost of an upload is roughly twice its size; a limit this path could not honour safely would not be a limit.
Downloads are always served as an attachment with nosniff, and a media type outside a known-inert set is served as application/octet-stream. The type is chosen by whoever uploaded the file, and a document served under its own type from this origin would run against the viewer's session.