vLLM has introduced Decode Context Parallelism (DCP), a technique that splits key-value caches across GPUs by sequence dimension rather than by attention heads. The approach sustains higher concurrency than baseline tensor parallelism, reaching 6,091 tokens per second per GPU at a concurrency of 512 while keeping KV cache usage at 82%, avoiding the memory walls that traditional methods hit at similar loads. The technique is aimed at long-context agentic workloads and is already integrated into vLLM, configurable via the decode_context_parallel_size parameter.
