IntBMoE is a Mixture-of-Experts design that decouples per-token participation, compute cost, and memory cost by pairing dense expert composition with sparse block execution, using a lightweight hypernetwork to merge a small learned codebook of expert blocks per layer. This gives every token full participation across the expert pool while keeping execution and materialization bounded, and a Dual-Path Residual Gating mechanism further couples two independently composed paths. Beyond gains on image classification, language modeling, and recommendation benchmarks, IntBMoE is deployed in AMap’s generative recommendation system serving hundreds of millions of users under a 60ms latency budget, delivering a 2.4% relative online A/B testing gain.
