Researchers introduce WHALE, a method that jointly optimizes both a language model’s weights and its harness code, the executable logic managing context and control flow, by alternating between fine-tuning the model via online rejection-sampling with the harness fixed, then searching for improved harness implementations given the updated weights. Tested on Qwen models across three domains, including question answering and mathematical reasoning, WHALE outperformed baselines that optimize only weights or only the harness by 4-24 percentage points.
