If your business website sits behind Cloudflare – and a large share of the web does, often through your hosting provider – a change is coming that decides who gets to use your content. Cloudflare has announced that from 15 September 2026 it will block, by default, web crawlers that both index sites for search engines and harvest the same pages to train AI models.
Until now, blocking AI crawlers was something you had to switch on yourself. Under the new policy, new customers and websites will automatically block AI training and AI-agent use on pages that carry ads, while still allowing normal search indexing – so your Google ranking is not the trade-off. Cloudflare’s CEO Matthew Prince put the reasoning bluntly: now that the majority of internet traffic is non-human, the web needs a sustainable deal between sites and the AI companies reading them.
There is a carrot as well as a stick. Cloudflare’s revamped Pay Per Use marketplace is designed to compensate site owners when their content shows up in AI chatbot answers, with early partners already signed on. Your website copy, guides and case studies are assets – this is the first mainstream mechanism that treats them that way.
What this means for your business
- Find out if you’re on Cloudflare. Many Australian small businesses are without knowing it, via their web host or agency. Ask, or check your DNS.
- Review your settings before the deadline. Existing free-plan sites in particular should confirm their crawler settings before 15 September 2026, so the default change doesn’t surprise them either way.
- Make a deliberate choice. Some businesses want AI assistants quoting their content (free exposure); others would rather block it or be paid. Either is fine – what’s costly is not deciding.
Worried this affects your business? Get a free 15-minute IT check – call Trends IT on 0485 011 911 or visit /contact/.