Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.
Magnitude is an open-source inference server, meaning software that runs language models and answers requests on a user's own computer. It profiles available hardware, recommends suitable local models, and connects them to coding agents for private and offline use on macOS, Linux, or Windows through WSL. Its catalogue entries help agents set up and use Magnitude.
This repository also configures its own agents. See what magnitude tells them →
Skill Claude CodeCodex
Operate Magnitude local inference through its headless CLI, recommend hardware-fit local models, monitor acquisition and loading, and connect an agent harness. Use for Magnitude service, model, catalog, setup, or harness requests.