Google explains CDN migration risks for Googlebot

Google published guidance on 11 November 2026 about moving a website to a Content Delivery Network, or CDN. It explains why crawler traffic can rise during the move, how a cold cache can put pressure on the origin server, and why broad bot controls may block Googlebot.

What happens during a CDN move?

A CDN distributes website assets through edge servers located closer to users. Google says this can reduce delays for visitors and protect the origin server from heavy traffic once cached files are available at the edge.

When DNS points to a fast CDN network, connection and response times can fall sharply. Google’s crawling systems may read those faster and more consistent responses as a sign that the website can handle more requests. Where there is crawl demand, the crawl capacity limit can then rise gradually.

The complication is that a new CDN starts with an empty cache. The first request for an uncached HTML document, CSS file, JavaScript file, image or other asset must reach the origin server. The edge server can then store a cacheable copy and handle later requests for that URL.

Why can the origin server remain busy?

Visitors usually concentrate on a smaller group of popular pages, such as a homepage, leading articles or current promotions. These pages can enter the CDN cache quickly. Crawlers, however, systematically work across a broader collection of URLs.

Faster edge responses can therefore encourage more crawling while each newly requested URL still causes a cache miss. Google says crawlers do not increase their rate immediately, and many CDNs include connection pacing, but the origin server can still do much of the work while the edge caches are filling.

Broad bot protection can create another search issue. Rules that treat every automated request alike may send Google’s crawlers to CAPTCHA challenges they cannot pass. This can leave the crawlers on an error or challenge page instead of the site’s content.

If that blocking stays in place, Google’s crawlers may reduce their activity because the website appears restricted or broken. Google says pages and shopping listings can then begin to disappear from Search.

Prepare the cache and bot settings

Google recommends preparing for the period before edge caches are populated. The aim is to control the load on the origin server without blocking crawlers that should be able to access the site.

• Allow verified bots through CDN and firewall settings where bot blocking is required.
• Use robots.txt or targeted rules for unwanted bots instead of blocking every automated client.
• Pre-warm high-traffic, frequently crawled and upcoming peak-traffic pages through the CDN at a controlled rate.
• If crawl volume creates critical origin load, temporarily return a 429 or 503 status for a few hours, or for one to two days at most.

Google says most major CDNs and web application firewalls provide a setting or managed rule for known or verified bots. It also provides published IP ranges that can be used to identify its crawlers. Businesses should check that CAPTCHA, firewall and web application firewall rules do not place verified Google crawlers behind challenge pages or 403 responses.

Review the effect on search access

A CDN move should include checks of origin load, cache behavior and crawler access. Logs can show whether requests are reaching the origin because of cache misses, while CDN and firewall settings can show whether verified crawlers are being challenged or blocked.

Businesses planning a CDN change can include these checks in an SEO review. A free audit is also available as a starting point.