fast-litellm: Instructions file for Claude Code

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Fast LiteLLM is a high-performance Rust acceleration layer for LiteLLM that provides drop-in replacements for performance-critical components. It uses PyO3 to create Python extensions from Rust code, achieving 2-20x performance improvements while maintaining full compatibility with the existing Python API.

Build Commands

Initial Setup

# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create virtual environment and install dependencies
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install maturin and development dependencies
uv add --dev maturin

# Development build (creates editable install)
uv run maturin develop

# Or use the automated setup script
./scripts/setup_dev.sh

Building

# Development build (fast, with debug symbols)
uv run maturin develop

# Release build (optimized)
uv run maturin build --release

# Build wheel for distribution
uv run maturin build --release --out dist

# Build for specific Python version
uv run maturin build --interpreter python3.11

Testing

# Run all tests
uv run pytest tests/

# Run specific test file
uv run pytest tests/test_accelerator.py

# Run single test
uv run pytest tests/test_basic.py::test_import_package

# Run tests with coverage
uv run pytest tests/ --cov=fast_litellm --cov-report=html

# Run tests excluding slow benchmarks
uv run pytest tests/ -m "not slow"

# Run only integration tests
uv run pytest tests/ -m integration

# Run Rust tests
cargo test

# Run Rust tests with specific module
cargo test --lib core

Code Quality

# Python formatting
uv run black fast_litellm/ tests/

# Python import sorting
uv run isort fast_litellm/ tests/

# Python linting
uv run flake8 fast_litellm/
uv run mypy fast_litellm/
uv run ruff check fast_litellm/

# Rust formatting
cargo fmt

# Rust linting
cargo clippy -- -D warnings

# Run all quality checks
uv run black --check . && uv run isort --check-only . && uv run flake8 . && uv run mypy fast_litellm/ && cargo fmt -- --check && cargo clippy