Cloudflare has introduced expanded controls for managing AI web traffic, replacing its previous all-or-nothing “Block AI Bots” setting with three separate categories: Search, Agent, and Training. Starting September 15, 2026, new default settings will block Training and Agent crawlers on pages that carry advertising while continuing to allow Search bots, with site owners able to adjust the settings themselves. Cloudflare also launched BotBase, an enterprise feature that gives site owners visibility into how verified bots are classified and how they behave. The changes follow a broader push by Cloudflare to give publishers more control over how their content is used by AI companies.
