ai

Butterbase AI Gateway

Every app has an LLM gateway with chat, embeddings, model listing, configuration, and usage reporting. One umbrella tool: manage_ai.

Action	What it does	Returns
chat	Synchronous chat completion (no streaming)	OpenAI-shaped { choices: [...] }
embed	Vector embeddings for string or string[]	OpenAI-shaped { data: [{ embedding: [...] }] }
list_models	Available models with capabilities	{ models: AiModel[] }
get_config	Current AI config (default model, BYOK key flag, etc.)	AiConfig
update_config	Set defaults, allowed models, max tokens, BYOK	AiConfig
get_usage	Token + cost aggregate over a window	usage record

---

1. Chat

manage_ai({
  action: "chat",
  app_id,
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user",   content: "What's RAG?" }
  ],
  model: "openai/gpt-4o-mini",     // optional — falls back to app's default
  temperature: 0.2,                // optional
  max_tokens: 500                  // optional
})

This action sets stream: false deliberately — agent tools don't stream. If you need partial-token deltas, drive the SDK's ai.chatStream(…) from inside a function or DO instead.

messages[].content can be a string or an array of content parts ({ type: "text", text }, { type: "image_url", image_url: {...} }, { type: "video_url", video_url: {...} }).

---

2. Embed

manage_ai({
  action: "embed",
  app_id,
  input: "hello world",            // or ["a", "b", "c"]
  model: "openai/text-embedding-3-small",   // optional
  encoding_format: "float"          // or "base64"
})

---

3. List models

manage_ai({ action: "list_models", app_id })
// → { models: [{ id, provider, capabilities: ["chat", "embed", ...], context_window, pricing }, ...] }

Use this to discover what the app can call — capabilities + context window matter when picking a model.

---

4. Configure