IBM Research argues that routing requests among multiple LLMs in agentic systems is more complex than simple task classification, since real-world cost depends heavily on caching behavior and infrastructure rather than list pricing alone. Researchers found Claude Sonnet 4.6 was cheaper in practice than GPT-4.1 despite higher sticker pricing due to superior cache performance, and built an optimization-based router that jointly balances cost, quality, and latency. The resulting system achieved configurations such as 84% accuracy for $93 while cutting costs 21% compared to routing everything to a single model.
