Skip to content

Chat ​

Every model you join answers on POST /v1/chat/completions, the request OpenAI clients send. This page covers sending a message, how hard a model reasons, and sending images, audio or video for it to read. For voice, see Speech and voice.

Sending a message ​

bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:1b","messages":[{"role":"user","content":"Hello!"}],"max_tokens":200}'

See the API reference for streaming, sessions, and the node/metrics endpoints.

Reasoning ​

Models that think expose two independent controls: thinking decides whether the model reasons at all, reasoning_effort how hard.

bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3.8:27b","messages":[{"role":"user","content":"Why is the sky blue?"}],
       "max_tokens":2048,"reasoning_effort":"low"}'

Ask GET /api/models which levels a model takes — reasoning_effort_levels is empty for models with no reasoning control, and the dashboard renders no selector for those rather than a disabled one. "none" is accepted everywhere and means the same as thinking off. An unsupported level is a 400 listing what the model does accept, never a silent downgrade. The model's thinking comes back in reasoning_content, separate from the answer in content.

dppan run takes the same controls locally as --think / --no-think and --reasoning-effort.

Reasoning is spent from the same max_tokens budget as the answer, so a model that thinks past the budget returns its reasoning and an empty answer. If replies come back blank on a reasoning model, raise MAX_TOKENS on the orchestrator (default 512) before looking elsewhere.

Images, audio and video as input ​

Send media to a model and it answers in text: a description, a transcription, analysis, or OCR.

Supported today: Gemma 4 · Gemma 3 · Qwen2-VL · Qwen2.5-VL (3B–72B) · Qwen3-VL (2B–32B, 30B-A3B, 235B-A22B) · GLM-4.6V-Flash · DeepSeek-OCR · Ultravox · Qwen3-ASR. See Models for the full matrix.

Images, audio and video sent in chat are read on the orchestrator; the other machines in the cluster never receive them.

For a voice clip or a transcript, see Speech to text.

Video: two ways to send it ​

input_video takes either frames or file, never both.

framesfile
What you sendimages you extracted, with timestampsa whole video file
Needs on the hostnothingffmpeg — see below
Containersn/amp4, mov, webm, mkv
Ceiling16 frames per part240 frames, 8 MiB encoded

Use frames to cover a long video sparsely (a ten-minute video works), and file for a short clip in full. The dashboard and dppan run --video-frames use frames, so they work everywhere.

Optional: ffmpeg for server-side video decode ​

Sending a whole video file needs ffmpeg and ffprobe on the orchestrator host's PATH. dppan does not ship them:

bash
brew install ffmpeg          # macOS
sudo apt install ffmpeg      # Debian / Ubuntu
sudo dnf install ffmpeg      # Fedora / RHEL
ffmpeg -version && ffprobe -version

They must be on the PATH of the orchestrator process, not just your shell — under systemd, set Environment=PATH= in the unit file or install to /usr/bin. GET /api/models reports "supports_video_files": true when they are found; without them, a video file is refused with that reason.

How much video fits depends on resolution, not file size:

ResolutionFramesAt the default 4 fps
512×51285~21 s
720×48064~16 s
1280×72024~6 s
1920×108010~2 s

For anything longer, downscale it first or send frames.

As a job, not a chat turn ​

The same reading can run as a queued understanding job (modality: "text" on the generation API) instead of a streamed chat reply:

bash
curl -X POST http://localhost:8001/v1/generations \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3vl:4b","modality":"text",
       "input_artifacts":["66815206-..."],
       "params":{"prompt":"What is the total on this receipt?"}}'

The answer arrives as a text/plain artifact. A job survives a restart and can be polled later, which suits batches (a folder of receipts) and slow reads; it takes one prompt and answers once, so it is not for conversations. It shares the queue with speech, music, images and video.

Which models can do this, and every error code, are in REST API → Understanding.

From the command line ​

Download the model's projector sidecar once, then attach image/audio files or ordered video frames to standalone local inference:

bash
dppan pull hf:owner/model:Q4_K_M --projector-only
dppan run --model hf:owner/model:Q4_K_M \
  --image photo.jpg \
  --audio question.wav \
  --video-frames frame-0.jpg@0 \
  --video-frames frame-1.jpg@1000 \
  --prompt "Describe these inputs"

--image, --audio, and --video-frames are repeatable. Projectors are discovered from their exact pull metadata; use --projector /path/to/mmproj.gguf for a custom location. In interactive mode, command-line attachments belong to the first turn and remain in later conversation history.

In the dashboard, the paperclip offers only what the selected model can read. Attachments are kept in your browser with the conversation, so a saved chat still shows them after a reload.

Free to run · Proprietary