Formulates three regularity conditions (Completeness, Prefix Consistency, Neutrality) that uniquely define token-level credit in RL for LLM post-training, providing a unified theoretical basis connecting on-policy distillation, RLOO, and GAE-based methods. Using this framework, the authors derive Policy Aligned Critic Training (PACT), which applies an Actor-then-Critic update order with importance-sampling correction to better align the critic with the updated policy. PACT achieves 72.87% average accuracy across four agentic math reasoning benchmarks (beating GRPO and PPO by 8.8 and 13.2 points) and a 67.4% pass rate on SWE-bench Verified.
