litellm-vertex-proxy-repair

LiteLLM Vertex Proxy Repair

A battle-tested repair skill for Toby-style local LiteLLM deployments on macOS.

Use this when a LiteLLM proxy backed by Vertex AI stops serving http://127.0.0.1:4000/v1, when Admin UI fails, or when you need to separate API recovery from PostgreSQL/Prisma recovery.

Quick start

1. Confirm whether the proxy is actually listening on port 4000.
2. Distinguish API-only outage from full-mode DB/UI outage.
3. Check whether Python is inheriting a macOS system proxy such as http://127.0.0.1:1082.
4. If Prisma or Admin UI is involved, force loopback bypass with NO_PROXY=127.0.0.1,localhost,::1.
5. Split the service into lite and full modes so API recovery does not depend on PostgreSQL/Prisma/Admin UI.
6. Verify /, /v1/models, and optionally /ui/login/ separately.

---

Deep Dive: API-Only vs Full DB Mode & Prisma Migrations (双语解析)

The Problem (问题背景)

English: LiteLLM running in "API-only" mode lacks database configuration (DATABASE_URL). While the /ui/login/ page might still load, submitting a login crashes the backend with Authentication Error, Not connected to DB! because session and user validation require a database. Furthermore, simply connecting a database isn't enough; if Prisma baseline migrations are skipped or interrupted, the backend background jobs will crash with errors like relation "LiteLLM_SpendLogs" does not exist.

中文: LiteLLM 在“纯 API 代理”模式下运行时，没有配置数据库（DATABASE_URL）。虽然你可以打开 /ui/login/ 页面，但在提交登录时，后端由于需要校验用户和生成 session，会直接崩溃并提示 Not connected to DB!。此外，仅仅连上数据库是不够的；如果跳过或中断了 Prisma 的基线同步（Baseline Migration），后端的统计进程就会不断报错崩溃，例如提示 relation "LiteLLM_SpendLogs" does not exist（缺少对应的表或视图）。

The Repair Strategy (修复策略)

English:

1. Database Setup: Install and run local PostgreSQL. Inject DATABASE_URL and DIRECT_URL into .env.
2. Prisma Dependencies: Reinstall LiteLLM with proxy extras (uv tool install --reinstall 'litellm[proxy,extra-proxy]') and execute prisma generate to build the local ORM client.
3. Lite vs. Full Mode Architecture: Implement a fallback toggle (LITELLM_MODE).
   • full enables the DB and UI console.
   • lite strips DB variables on startup, reverting to a pure API proxy to ensure AI generation stays online even if the database fails.
4. Full Schema Migration: Run Prisma baseline migrations to synchronize all 100+ tables and views (including LiteLLM_SpendLogs). This resolves the missing relation crashes and fully restores the UI backend.