ModelSaar

Get started

From nothing to a first request routed through auto. Two paths: the beginner path installs, configures and verifies in a few commands; the expert path is the same thing without the wrapper.

Status: ModelSaar is a public alpha. No release has been tagged and no package is published, so every install path below pins commit d6e09e770da6 of the hardening/review-2026-10-11 branch — the candidate described on the roadmap (two independent reviews reproduced and re-checked its defects; independent verification of this exact candidate is pending). Re-running the installer later moves you to the next pinned commit.

Before you start

1. Install

Beginner path: one command

curl -fsSL https://modelsaar.com/install.sh | sh

The script is short and does only this: checks your platform and Python; installs the ModelSaar package at the pinned commit with uv tool install, pipx or python3 -m pip install --user (first one found; it does not install those tools for you); runs modelsaar init if you have no config yet (it looks for Ollama on 127.0.0.1:11434 and other documented loopback ports, writes ~/.modelsaar/config.yaml and private keys with mode 600); then starts a temporary gateway, sends one request through auto, stops it and prints the result with your next step. Existing config, keys and state are never overwritten; an occupied gateway port is reported, not fought over.

To read it before running it:

curl -fsSL https://modelsaar.com/install.sh -o install.sh && less install.sh && sh install.sh

Options are environment variables: MODELSAAR_ENDPOINT=http://host:port (backend other than local Ollama), MODELSAAR_PORT=8484, MODELSAAR_NO_CHECK=1 (skip the verification request), MODELSAAR_REF=<sha> (install a different commit — you take responsibility for it).

Expert path: install by hand

With uv (recommended; isolated, fetches Python 3.12 if needed):

uv tool install git+https://github.com/gopalasubramanium/modelsaar@d6e09e770da6e8bd560da7132cb263eb02ec7151

With pipx:

pipx install git+https://github.com/gopalasubramanium/modelsaar@d6e09e770da6e8bd560da7132cb263eb02ec7151

With plain pip (user install; needs Python ≥ 3.10 and ~/.local/bin on your PATH):

python3 -m pip install --user git+https://github.com/gopalasubramanium/modelsaar@d6e09e770da6e8bd560da7132cb263eb02ec7151

From a source checkout (what the test suite and the verification scripts use):

git clone https://github.com/gopalasubramanium/modelsaar && cd modelsaar && git checkout d6e09e770da6e8bd560da7132cb263eb02ec7151 && python3 -m venv .venv && . .venv/bin/activate && pip install -e .

All four give you the modelsaar command and nothing else; nothing starts automatically.

2. Configure and run

Before this step an inference server with at least one model must be running (for example ollama serve after ollama pull …); ModelSaar never downloads models.

If you used the one-line installer, modelsaar init has already run (the installer printed the models it discovered) — skip to modelsaar doctor. With uv, pipx, pip or a source checkout, run it once:

modelsaar init

Finds Ollama on 127.0.0.1:11434 (or pass --endpoint URL --type ollama|openai_compatible), writes ~/.modelsaar/config.yaml and keys, discovers your models. Refuses to overwrite an existing config without --force; after the installer, do not run it again.

modelsaar doctor

Reachability, assumed context lengths, unknown capabilities, cloud-forwarded aliases — with a remedy per warning.

modelsaar serve

Starts the gateway at http://127.0.0.1:8484/v1 (loopback only). Leave it running; stop with Ctrl-C.

Optional but recommended once: modelsaar probe --all verifies streaming, tool calling, JSON schema and image input on each model with short real requests (it uses your backend's compute). Run it while the gateway is up; the probes go through the gateway so they never collide with your own traffic.

Verify

modelsaar test

In a second terminal. Identifies the listener on the configured port (GET /healthz must answer the ModelSaar health contract before any key is sent), lists the models the gateway advertises and sends one chat completion through auto; prints the deployment that answered and the request id.

If it prints an HTTP 422, no model qualified — the body says which blocker applies to each deployment and what to change. If it cannot connect, the gateway is not running on the configured port; the message says so. If something else is listening on that port, it says "not a ModelSaar gateway" and sends nothing: change server.port or stop the other service.

3. Connect an app

Any app that speaks the OpenAI Chat Completions API needs three values. First print the key in a terminal:

head -1 ~/.modelsaar/keys/api_keys

Prints one line starting with msk_. That printed line is the API key to paste into a GUI field; the command itself is shell syntax, not a key.

Base URL:  http://127.0.0.1:8484/v1
Model:     auto        # or auto-fast, auto-code, auto-vision, or a concrete deployment id
API key:   msk_…  (the line printed by the command above)

Or let ModelSaar print the exact settings for your app — one command per app:

--apply merges file-based configs with a backup and keeps the file's permissions. The key never leaves your machine unless you send it somewhere.

Addresses when the app is not on the same machine. 127.0.0.1 always means "the machine (or container) where the app runs". An Open WebUI container on the same host reaches the gateway at http://host.docker.internal:8484/v1; an app on another computer needs the gateway bound beyond loopback (server.host: 0.0.0.0 and security.allow_lan: true, with TLS or a private network in front — see Privacy). modelsaar client-config … --docker prints the container form.

One model only? auto uses it whenever it qualifies; nothing else changes. More models later expand its choices without touching the app.

Running it day to day

Keep it running
Run modelsaar serve under your service manager; a launchd plist and a systemd unit are in examples/. Docker Compose is documented in Installation (gateway only; models stay with your runtime).
Stop / restart
Ctrl-C in the terminal, or your service manager's stop/restart. Configuration changes need a restart; in-flight requests finish on the old config.
Update
Re-run the installer (it pins the newer candidate commit), or with uv/pipx/pip re-run the install command with the new commit. Then modelsaar config validate and modelsaar doctor. Keys and state are untouched; evidence from an older evidence scheme is kept but not used until you re-probe.
Troubleshoot
modelsaar doctor first; then modelsaar decisions --limit 10 for the last routing records and modelsaar explain -m "…" [--tools --image --json-schema] to dry-run a request. More in Troubleshooting.
Uninstall
modelsaar uninstall removes only the files ModelSaar created under ~/.modelsaar (config, its backups, keys, state DB) and lists anything it left in place. Then uv tool uninstall modelsaar, pipx uninstall modelsaar or pip uninstall modelsaar. Models and client configurations are never touched; applied client configs have .bak-<timestamp> copies next to them.

What has actually been tested

Two kinds of evidence, stated separately. At the current pinned commit the installer and the CLI were exercised end to end on Linux with the package-manager step stubbed (fresh install, repeat install, an unrelated service on the port, a running gateway, serve, test, one client configuration — docs/live-runs/2026-10-12-installer-scenarios/). At an earlier pin (80276bb) the real installer paths — uv, pipx and pip --user on Linux x86_64, uv on macOS arm64 against the owner's Ollama host — were executed with real package installation (docs/live-runs/2026-10-11-installer/). A real package-manager installation at the current pin, and a real-client walkthrough, have not been run yet; both are listed in the verification report. WSL2 follows the Linux path and has not been executed by the project. Native Windows is not supported.