Researchers present Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125 billion total parameters that activates only 6 billion per token, achieving competitive pre-training performance while using roughly one-third the activated parameters and one-ninth the training compute of its 397 billion parameter predecessor. The architecture combines Gated DeltaNet and global attention in a hybrid token-mixing design, supplemented by n-gram embedding tables stored off-accelerator and a Gated Residual design with four residual stream branches. The team finds that loss and downstream accuracy do not always move together, and that architectural changes paired with the Muon optimizer jointly improve training stability and efficiency.
