A paper posted to arXiv on October 16, 2026 by Shuyang Xiang proves a “conditional no-hair theorem” for autoregressive transformers, showing that an initial prompt’s influence on a model’s hidden states decays roughly as a power law as more tokens are generated, with trained models forgetting faster than untrained ones. The author derives an upper bound on this decay for finite-depth pre-LayerNorm softmax transformers with bounded attention logits, and shows experimentally that artificially canceling the dilution effect prevents the forgetting, confirming dilution as the mechanism. The result implies that once enough text has been generated, any surviving property of the original prompt, such as its language, must be carried forward through the generated text itself rather than the prompt directly.
