omni-inference

Overview

The core OpenAI-compatible inference endpoints: chat completions, embeddings, images, audio (TTS/STT), moderations, rerank, and the Responses API. The primary integration surface for AI agents.

Authentication

All requests require a valid Bearer token or session cookie. Obtain a token via POST /api/auth/login or configure REQUIRE_API_KEY=false for local development.

Endpoints

POST /api/v1/chat/completions

Create chat completion

OpenAI-compatible chat completions endpoint. Routes to configured providers.

curl -X POST https://localhost:20128/api/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
  -H "Content-Type: application/json" \
  -d '{}'

GET /api/v1/ws

Chat completion over WebSocket (handshake + upgrade)

OpenAI-compatible chat over a WebSocket connection. GET with ?handshake=1 returns the connection descriptor (auth path, message protocol and live-event channels) as JSON; a plain GET without an Upgrade returns 426 Upgrade Required. After upgrading, the client exchanges JSON frames — {type:"request", id, payload:{model, messages}} to start a completion and {type:"cancel", id} to abort it. A separate live channel (default port LIVE_WS_PORT=20129, path /live) streams dashboard events on the requests, combo and credentials topics with a 15s heartbeat. Requires an API key.

curl https://localhost:20128/api/v1/ws \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"

POST /api/v1/providers/{provider}/chat/completions

Create chat completion (provider-specific)

Routes to a specific provider by name.

curl -X POST https://localhost:20128/api/v1/providers/{provider}/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_TOKEN"
  -H "Content-Type: application/json" \
  -d '{}'

POST /api/v1/api/chat

Ollama-compatible chat endpoint

Provides compatibility with Ollama's /api/chat format.