Researchers propose register tokens — dedicated fixed-position tokens whose hidden states are trained to carry reasoning progress — to let masked diffusion language models continue multi-step reasoning across generation chunks after earlier text has been cleared from context, instead of requiring the full text history to remain in context. Models are post-trained to decode a chunk, discard it while preserving the register values, and resume decoding from the prompt plus carried state. On LLaDA and Dream, register-token carry outperforms discrete-text carry on every benchmark tested, with gains of up to 8.5 points on math and 19.5 points on code, and the approach can be further refined with reinforcement learning on long-horizon tasks.
