Integrations
What connects to ModelSaar, what ModelSaar connects to, and the evidence behind each line. Product names identify compatible software; no partnership or endorsement is implied.
| area | what | evidence | notes |
|---|---|---|---|
| Interface | OpenAI-compatible Chat Completions (/v1/chat/completions, streaming SSE), /v1/models, /healthz, /readyz | fixture tests live | Responses API and Anthropic Messages return an explicit 501. |
| Interface | /v1/embeddings with identity-anchored alias contracts | fixture tests | Real-model embeddings NOT RUN; contract, identity anchoring, deadlines and output validation are fixture-tested only. |
| Backend | Ollama (discovery via /api/tags, /api/show, /api/ps; inference via its OpenAI-compatible layer) | fixture tests live | Ollama 0.32.13 with 8 models on an ASUS GX10, 2026-10-11. |
| Backend | Generic OpenAI-compatible server (vLLM, llama.cpp server, LM Studio, …) with declared models; vLLM /tokenize used for token counts | fixture tests | Not run against a real vLLM/llama.cpp server yet. |
| Client | Open WebUI | documented | Configuration generated by modelsaar client-config open-webui; real client workflow not run. |
| Client | OpenClaw | documented | modelsaar client-config openclaw; real client workflow not run. |
| Client | OpenCode (V1 and V2 config schemas) | documented | modelsaar client-config opencode (V1) or modelsaar client-config opencode-v2 (V2); real client workflow not run. |
| Client | Hermes Agent | documented | modelsaar client-config hermes; real client workflow not run. |
| Behaviour | Tool calls and tool-result continuation on the same deployment; session pinning via X-ModelSaar-Session / X-OpenWebUI-Chat-Id | fixture tests live | Live: direct HTTP checks (scripts/live_client_checks.py), not a client's own workflow. |
| Behaviour | Strict JSON schema, image input and context-size routing; unsupported media rejected before dispatch | fixture tests live | |
| Behaviour | Pool memory budgets (incl. assumed residency between polls), uncertain-occupancy holds on stream/probe/embedding timeouts, probes under gateway admission | fixture tests | Not exercised on real hardware; the live run predates the 2026-10-12 corrections. |
| Platform | macOS (arm64) and Linux (x86_64): source install, gateway, probes, evaluation | live | Windows: use WSL2; native Windows is untested. |
| Platform | Docker Compose (gateway only; models stay with your runtime) | fixture tests | Fresh-volume start is checked by scripts/compose-smoke.sh (defined in CI; no CI run exists yet for the current revision). |
live exercised against a real backend or on real hardware, with artefacts in the repository; fixture tests automated tests in CI against a controlled fake backend that implements the Ollama and OpenAI-compatible protocols; documented configuration written from the client's primary documentation on 2026-10-11 and generated by modelsaar client-config, not yet run as a real client workflow. The verification report lists every item still marked NOT RUN with the command that would run it.
Apps (clients)
Any application that speaks the OpenAI Chat Completions API can use ModelSaar: set the base URL to the gateway, the API key to your generated key, and the model to auto (or another alias). The clients below have generated, documented configuration; each page says exactly what was verified.
Open WebUI
Admin Panel → Connections → OpenAI connection with URL http://127.0.0.1:8484/v1 (from a container: http://host.docker.internal:8484/v1). Models are auto-listed from /v1/models. ENABLE_FORWARD_USER_INFO_HEADERS or a custom X-ModelSaar-Session header keeps a chat on one model; TASK_MODEL_EXTERNAL=auto-fast sends background tasks to the fast alias.
OpenCode
A provider entry using @ai-sdk/openai-compatible with the aliases as models; both the V1 (provider) and V2 (providers) config schemas are generated, and --apply refuses to mix them.
OpenClaw
Provider configuration pointing at the gateway; the client keeps its own tool execution and permissions.
Hermes Agent
YAML provider/model settings merged into Hermes's config with a backup; file permissions are preserved.
Scripts and SDKs
Any OpenAI SDK with base_url and api_key set; response headers carry the request id and the deployment that answered.
Inference backends
Ollama
First-class. Discovery uses Ollama's native API (/api/tags, /api/show for capabilities, context and template, /api/ps for what is loaded, /api/version); inference goes through Ollama's OpenAI-compatible layer so its tool-call semantics are preserved exactly. Cloud-forwarded models (remote_model) are detected and excluded under local_only. Known limits of that layer (tool_choice=required/named, remote image URLs, logit_bias, n, logprobs) are rejected with blockers instead of being altered. The served context of an unloaded model is unknown to Ollama's API; ModelSaar assumes 4096 unless the Modelfile, /api/ps or an override says otherwise, and doctor warns about every assumption. Verified live with Ollama 0.32.13 and eight models.
OpenAI-compatible servers (vLLM, llama.cpp server, LM Studio, …)
Supported through a generic adapter: /models for discovery, declared models in the config for what discovery cannot tell (context length, capabilities, tags, memory estimate, identity), vLLM's /tokenize for token counts when present. Tested against the fixture implementation only; not yet run against a real vLLM or llama.cpp server. Declare a locality for such a backend consciously — a generic endpoint exposes no evidence of where it runs.
Several backends and hosts
Backends can live on other machines (the recorded runs talk to a GX10 over a private network). Each backend declares a resource pool so concurrency and memory budgets are counted where the work happens. Routing across hosts is not scheduling: ModelSaar does not load, unload or move models.
Not supported yet
- Responses API and Anthropic Messages (explicit 501).
- Audio and video inputs, multimodal embeddings, OCR and guardrail pipelines (requests carrying them are rejected before dispatch).
- Managed model loading/unloading, cascades, ensembles, learned routing policies.
- Native Windows (use WSL2).
See the roadmap for what is planned and Extending for how to add a backend or client with its own tests.