OpenAI-compatible clients
Tools and SDKs written for OpenAI work with Inclavate: point them at your orchestrator.
| Base URL | http://<orchestrator>:8001/v1 |
| API key | Any value. Only the admin endpoints check a token |
| Model | The id a model was joined or installed with, such as llama3.2:1b or z-image-turbo; GET /v1/models lists them |
What works
| OpenAI endpoint | Details | |
|---|---|---|
POST /v1/chat/completions | Streaming or not, with images, audio and video as content parts | Chat and models API |
GET /v1/models | The models this deployment serves | Chat and models API |
/v1/files | Uploads, the same store as artifacts | Files |
POST /v1/audio/speech | Speech from text | Audio and images API |
POST /v1/audio/transcriptions | Text from a clip | Audio and images API |
POST /v1/images/generations | Images from a prompt | Audio and images API |
POST /v1/images/edits | Images from images | Audio and images API |
POST /v1/audio/music takes the same shape for songs, though it is not an OpenAI endpoint. OpenAI endpoints not listed here are not provided.
From Python
python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8001/v1", api_key="unused")
reply = client.chat.completions.create(
model="llama3.2:1b",
messages=[{"role": "user", "content": "Hello!"}],
)
print(reply.choices[0].message.content)client.audio.speech.create(), client.audio.transcriptions.create(), client.images.generate() and client.files.create() work the same way. What each model needs, such as a voice for speech, is on the Audio and images API page.
Open WebUI
- Chat: add an OpenAI API connection with the base URL
http://<orchestrator>:8001/v1and any key. The models you have joined appear in its model list. - Images: see Images › From an OpenAI client.