The paper introduces RRSI, a method for automatically refining LLM agent harnesses while mitigating overfitting during recursive self-improvement. It constrains the evolution process via a temporally annealed edit budget and a critic/pruner selection mechanism that favors reusable mechanisms over benchmark-specific fixes. Across eight benchmarks, RRSI achieved up to 14.1 points of in-distribution gain and 4.7 points out-of-distribution, while cutting policy token usage by 30%.
