litellm-rs: Skill for Claude Code

Provider Architecture Guide

Architecture Overview

Providers come in two tiers behind one implementation trait:

Tier 1 — Catalog-only (zero code). An OpenAI-compatible endpoint that differs only in base URL, auth env var, and advertised capabilities/models. It has no Rust module: one static entry in src/core/providers/registry/catalog.rs fully describes it, and the factory builds an OpenAILikeProvider from that data at runtime. This is the primary path for new integrations.

Tier 2 — Code-based. A provider needing custom request/response transformation, custom auth signing, non-standard streaming, or rich model metadata lives in src/core/providers/<name>/, implements LLMProvider, and is wired into routing.

Routing does not use trait objects. Router deployments store the closed Provider enum (src/core/providers/mod.rs), which dispatches to concrete provider structs. LLMProvider (src/core/traits/provider/llm_provider/trait_definition.rs) is the interface every variant implements — implementing the trait alone does not make a provider routeable; enum variant, dispatch arm, and factory wiring are crate-level changes. Trait objects appear only at the edges: the error mapper (Box<dyn ErrorMapper<ProviderError>>) and the streaming return type (Pin<Box<dyn Stream<Item = Result<ChatChunk, ProviderError>> + Send>>).

Enumerating Current Providers

Do not rely on memorized counts — they change often. Count from source:

# Tier 1: one def_chat()/def_local_chat() call per entry
# (each count includes the helper fn definition itself, so subtract 1)
grep -c 'def_chat(' src/core/providers/registry/catalog.rs
grep -c 'def_local_chat(' src/core/providers/registry/catalog.rs

# Tier 2: code-based provider modules (base/factory/macros/registry are infrastructure)
ls -d src/core/providers/*/ | grep -vE '/(base|factory|macros|registry)/'

Adding a Tier 1 Provider (Primary Path)

Two edits, nothing else:

// src/core/providers/registry/catalog.rs
def_chat(
    "myprovider",
    "My Provider",
    "https://api.myprovider.com/v1",
    "MYPROVIDER_API_KEY",
),