Researchers investigate whether the different functional operations that make up a language model’s chain-of-thought reasoning, such as problem formulation, goal decomposition, and deduction, have distinct geometric structure inside the model’s hidden representations. The study finds that these reasoning operations are separable in held-out representations, with separability peaking in the middle layers, and that this structure is not explained by lexical or positional confounds alone. Attention-masking experiments further show that operation-aligned representations depend on the preceding reasoning context, indicating that language models maintain a representational correspondence between the linguistic expression of reasoning and its internal geometric organization.
