Chat
Every model you join answers on POST /v1/chat/completions, the request OpenAI clients send. This page covers sending a message, how hard a model reasons, and sending images, audio or video for it to read. For voice, see Speech and voice.
Sending a message
curl -X POST http://localhost:8001/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama3.2:1b","messages":[{"role":"user","content":"Hello!"}],"max_tokens":200}'See the API reference for streaming, sessions, and the node/metrics endpoints.
Reasoning
Models that think expose two independent controls: thinking decides whether the model reasons at all, reasoning_effort how hard.
curl -X POST http://localhost:8001/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"qwen3.8:27b","messages":[{"role":"user","content":"Why is the sky blue?"}],
"max_tokens":2048,"reasoning_effort":"low"}'Ask GET /api/models which levels a model takes — reasoning_effort_levels is empty for models with no reasoning control, and the dashboard renders no selector for those rather than a disabled one. "none" is accepted everywhere and means the same as thinking off. An unsupported level is a 400 listing what the model does accept, never a silent downgrade. The model's thinking comes back in reasoning_content, separate from the answer in content.
dppan run takes the same controls locally as --think / --no-think and --reasoning-effort.
Reasoning is spent from the same max_tokens budget as the answer, so a model that thinks past the budget returns its reasoning and an empty answer. If replies come back blank on a reasoning model, raise MAX_TOKENS on the orchestrator (default 512) before looking elsewhere.
Images, audio and video as input
Send media to a model and it answers in text: a description, a transcription, analysis, or OCR.
Supported today: Gemma 4 · Gemma 3 · Qwen2-VL · Qwen2.5-VL (3B–72B) · Qwen3-VL (2B–32B, 30B-A3B, 235B-A22B) · GLM-4.6V-Flash · DeepSeek-OCR · Ultravox · Qwen3-ASR. See Models for the full matrix.
Images, audio and video sent in chat are read on the orchestrator; the other machines in the cluster never receive them.
- API: multimodal
messages[].contentparts — see REST API → Chat completions. - CLI:
--image,--audio,--video-frames— see From the command line.
For a voice clip or a transcript, see Speech to text.
Video: two ways to send it
input_video takes either frames or file, never both.
frames | file | |
|---|---|---|
| What you send | images you extracted, with timestamps | a whole video file |
| Needs on the host | nothing | ffmpeg — see below |
| Containers | n/a | mp4, mov, webm, mkv |
| Ceiling | 16 frames per part | 240 frames, 8 MiB encoded |
Use frames to cover a long video sparsely (a ten-minute video works), and file for a short clip in full. The dashboard and dppan run --video-frames use frames, so they work everywhere.
Optional: ffmpeg for server-side video decode
Sending a whole video file needs ffmpeg and ffprobe on the orchestrator host's PATH. dppan does not ship them:
brew install ffmpeg # macOS
sudo apt install ffmpeg # Debian / Ubuntu
sudo dnf install ffmpeg # Fedora / RHEL
ffmpeg -version && ffprobe -versionThey must be on the PATH of the orchestrator process, not just your shell — under systemd, set Environment=PATH= in the unit file or install to /usr/bin. GET /api/models reports "supports_video_files": true when they are found; without them, a video file is refused with that reason.
How much video fits depends on resolution, not file size:
| Resolution | Frames | At the default 4 fps |
|---|---|---|
| 512×512 | 85 | ~21 s |
| 720×480 | 64 | ~16 s |
| 1280×720 | 24 | ~6 s |
| 1920×1080 | 10 | ~2 s |
For anything longer, downscale it first or send frames.
As a job, not a chat turn
The same reading can run as a queued understanding job (modality: "text" on the generation API) instead of a streamed chat reply:
curl -X POST http://localhost:8001/v1/generations \
-H 'Content-Type: application/json' \
-d '{"model":"qwen3vl:4b","modality":"text",
"input_artifacts":["66815206-..."],
"params":{"prompt":"What is the total on this receipt?"}}'The answer arrives as a text/plain artifact. A job survives a restart and can be polled later, which suits batches (a folder of receipts) and slow reads; it takes one prompt and answers once, so it is not for conversations. It shares the queue with speech, music, images and video.
Which models can do this, and every error code, are in REST API → Understanding.
From the command line
Download the model's projector sidecar once, then attach image/audio files or ordered video frames to standalone local inference:
dppan pull hf:owner/model:Q4_K_M --projector-only
dppan run --model hf:owner/model:Q4_K_M \
--image photo.jpg \
--audio question.wav \
--video-frames frame-0.jpg@0 \
--video-frames frame-1.jpg@1000 \
--prompt "Describe these inputs"--image, --audio, and --video-frames are repeatable. Projectors are discovered from their exact pull metadata; use --projector /path/to/mmproj.gguf for a custom location. In interactive mode, command-line attachments belong to the first turn and remain in later conversation history.
In the dashboard, the paperclip offers only what the selected model can read. Attachments are kept in your browser with the conversation, so a saved chat still shows them after a reload.