The authors introduce BI-Bench, the first benchmark systematically testing whether LLMs can answer business-intelligence questions end-to-end (table discovery, transformation, joins, and analysis) using question/answer pairs mined from real BI dashboards, finding that even frontier LLMs score under 50% accuracy. To close the gap, they build BI-Agent, which decomposes BI workflows into subtasks like search, join, and transform and orchestrates specialized data-management methods, then post-trains it with trajectories synthesized from real BI projects using both SFT and RL. BI-Agent improves accuracy by up to 40 points over vanilla LLMs, with post-training contributing up to 30 additional points.
