The post describes a three-layer stack vLLM uses to optimize serving for agentic workloads: a data plane with hybrid KV cache management using packed layouts and hierarchical offloading to distributed storage; an execution plane applying model-specific parallelism strategies, such as decode context parallelism for Kimi K3 and data/expert parallelism for DeepSeek V4, paired with two-level scheduling to avoid head-of-line blocking; and a control plane that coordinates prefill/decode disaggregation through saturation profiling and rate-matching. The system favors session-aware routing that preserves cache locality over pursuing perfectly even load balancing, reporting up to 130K tokens per GPU-second in agentic serving scenarios.
