Amazon researchers describe Turnstile, a Rust-based proxy that sits between an agent harness and its inference backend to capture exact token IDs, log probabilities, and loss masks at the moment of generation, addressing the loss of information that occurs when only text transcripts are logged for reinforcement learning. Because it operates as a transparent proxy, Turnstile lets existing agent harnesses generate data suitable for RL training without any modification, and it supports multimodal inputs and mixture-of-experts routing architectures. The team reports steady improvement from RL training on Turnstile-captured data across both text-only coding agents and multimodal computer-use agents.
