Researchers ran a controlled scaling study comparing four retrieval-augmented generation paradigms across 28 corpus sizes ranging from 1.7 million to 601 million tokens. File-system-agent retrieval outperformed BM25 at smaller scales, but the study found BM25 overtakes it at every larger shared tier, leading by nearly 20 points at full scale. The analysis attributes this to lexical retrieval’s near scale-invariant query cost, in contrast to iterative exploration’s rising cost and graph-based methods’ prohibitive construction overhead as corpora grow.
