vllm-project/semantic-router

vLLM Semantic Router is a programmable routing layer that chooses or combines language models for each request in a system using multiple models and types of computing infrastructure. It helps teams route inference by signals such as user preferences, application policies, quality, cost, latency, privacy, and safety requirements. The catalogue skills and instructions support configuring and operating this model-routing system.