The vLLM AFD plugin introduces Attention-FFN Disaggregation, separating a Mixture-of-Experts model’s attention and FFN components into independently deployed services that can scale separately using different rank topologies and backend-specific connectors. It maintains vLLM’s existing request lifecycle while supporting synchronous connectors for decode throughput on GPUs and NPUs and asynchronous connectors for prefill operations, with reported throughput gains of up to 11.3% in certain configurations.
