A single bot toggle is three jobs. On September 15, Cloudflare will treat them as one.
Split Training from Search before Sept 15.
Most marketing sites still have one switch labeled something like Block AI bots. That switch was built for a simpler year, when the job was stop the scrapers. Cloudflare now splits the same traffic into Search, Agent, and Training (Cloudflare Blog). Search indexes you so someone can find you later. Agent fetches a page for a person waiting in a chat. Training absorbs the page into a model. Those are not the same operating decision.
On September 15, 2026, they stop being interchangeable.
The Mixed Crawler Rule Is The Whole Story
Cloudflare will apply the strictest rule that fits a crawler. If a bot does Search and Training, a Training block wins. Cloudflare names the mixed cases in the open: Googlebot, Applebot, and Bingbot (Cloudflare Blog). Search Engine Journal walked the same mechanic: a network-level block is not a robots.txt suggestion Google can ignore (Search Engine Journal). Playwire published publisher reports of 403s on Googlebot when Training is set to block, including cases where the dashboard itself showed Googlebot as blocked (Playwire).
This is not a ranking rumor. It is an HTTP response at the CDN.
If Googlebot gets 403 on your sitemap or your product templates, the rest of the search program is arguing with a closed door.
Do Not Mash The New Defaults Into The Mixed Rule
Two things happen on the same date. They are easy to mash together.
New domains, and free accounts that never touched the setting, get new defaults: Training and Agent blocked on pages that show ads, Search still allowed. That is a default for people who never chose.
The mixed crawler rule is different. It applies to anyone who already chose Block AI bots, or who set Training to block. You do not have to be a new customer. You already opted into the strictest rule. September 15 is when Cloudflare starts enforcing mixed purpose against that choice.
If you are not on Cloudflare, this is not your ticket. If you are, it is a this-week ticket whether or not you run ads on the page.
This Week: Three Columns, Three URLs, One Opt Out
Open the zone. Find AI crawler controls (Security, then bots or AI crawler policies, depending on the plan). Make three columns: Search, Agent, Training.
Write the policy in English first. Search stays allowed on every template that has to rank or get cited. Training is a board choice, not a leftover checkbox. Agent is a conversion choice: if a chat fetch bot is how a buyer pulls the spec sheet, blocking it is a demand decision.
Then prove it. Fetch three money URLs as Googlebot: home, the highest-intent product or offer page, and the sitemap. You want 200, not 403. Screenshot the response codes into the same doc as the three-column policy.
If you need Training blocked and Googlebot open, use Cloudflare's opt-out for the mixed-crawler default before September 15. The company says that opt-out is available to every customer until the date (Cloudflare Blog). Do not wait for a ranking dip to discover you needed it.
Tell The Board A Channel Story, Not A Bot Story
The sentence for leadership is not we blocked AI. It is: we kept Search crawling, we decided Training on purpose, we checked Googlebot on the pages that close revenue.
Annotate September 15 in Search Console. If impressions on a money URL fall after that week, look at crawl, not at copy first.
A modern marketing site treats crawler policy as part of conversion architecture. The page that cannot be fetched cannot convert, get cited, or rank. Sept 15 is two weeks out. Split the jobs this week.
