The paper studies how large language models retrieve and use internal knowledge by performing layerwise interventions on hidden states across Qwen, Llama, and Gemma models answering country-continent questions. The authors track two internal signals, pair-conditioned request direction and global request direction, across layers to disentangle routing (finding the right knowledge) from content use. Results show routing-information dependence declines in later layers while content dependence persists, with the specific timing of this shift varying by model architecture.
