model-routing-selector

Model Routing Selector

Overview

The horizontal workflow graph (Chapters 5-6) decomposes the agent into specialized nodes: a query analyst, a retrieval strategist, a synthesis node, a validation node. Most teams start with the same frontier model powering every node. It works, and it is wildly expensive.

Selective intelligence matches model capability to task complexity. Running a 3B SLM can be 10-30x cheaper per token than its 405B sibling; at thousands or millions of invocations per day, that difference decides whether the project survives its first budget review. The chapter's key insight is that most nodes do not need frontier reasoning — a fine-tuned 3B model classifies alerts, a 3.8B Triplex model beats GPT-4o at knowledge-graph construction, and only the open-ended synthesis and causal-reasoning nodes (10-20% of invocations) reliably benefit from a frontier model.

This skill re-derives the book's DevOps assignment (Example 8-13) from a node's required_quality bar and a per-model cost/capability catalog, so you can see why each node gets the model it gets rather than accepting an opaque config.

When to Use

• A pipeline runs every node on one frontier model and cost is unsustainable.
• You are adding a node and need to know the cheapest model that clears its bar.
• A node's per-query difficulty varies and you are deciding static vs cascade.
• You have production traffic and want to justify a learned router.

Phrases that should invoke this skill: "which model for this node", "selective intelligence", "cheapest model that meets the bar", "static vs cascade routing", "cut inference cost", "RouteLLM", "FrugalGPT cascade".

When NOT to Use

• Single-model systems. With one node type and one model there is nothing to route.
• Choosing the graph data model (property graph vs RDF vs hypergraph) — that is graph-model-selector (Ch3).
• Measuring a routing policy after deployment — that is cost-performance-scorer (Ch8). This skill decides routing before you run; that skill scores what actually happened.
• KV-cache / latency budgeting — that is kv-cache-latency-budgeter (Ch8). Routing lowers cost per weight; it does not lower the concurrency ceiling.