Researchers present Periscope, a training-free inference method that lets a frozen language model answer questions over text far longer than its context window. The method arranges a document’s chunks on a grid and asks the model the same question over local spans and strided spans that sample the whole text, reading the log-odds of each answer in a single pass to build a cost-free evidence map pointing to the relevant chunk. On LongBench v2, reading only the top-ranked 9,000 tokens identified by the map matched the same model’s best full-window read across window sizes from 32,000 tokens to 1 million tokens, and it outperformed the best window read by 5 points on InfiniteBench, where the median context is 150,000 tokens. The approach also sharply cuts memory needs, letting a 27B-parameter model process 4.5-million-token contexts on a single 80GB GPU instead of the 296GB a direct pass would require.
