LangChain benchmarked NVIDIA’s open-source NeMo Switchyard model-routing library against its Deep Agents evaluation suite of 145 multi-step agentic tasks. Calls were routed between a frontier model, Claude Opus 4.8, and a smaller 30-billion-parameter model, Nemotron 3.5 Lightning, using escalation-mode routing, where tasks start on the cheaper model and escalate after two consecutive negative verdicts from a judge model. The setup handled 93% of requests with the smaller model and needed the frontier model for only 7%, cutting total cost by 74% at a 6-point accuracy reduction versus using Opus exclusively. LangChain notes the judge model itself accounted for roughly 21% of routed spend, making it a prime target for further optimization.
