Crawl Budget

Crawl budget is the number of pages Googlebot will crawl on your website within a given period. Google does not crawl every page of every site constantly — it allocates a certain amount of crawling attention based on how large, important, and fresh it judges your site to be.

Two things shape it: crawl capacity (how much Google can crawl without overloading your server) and crawl demand (how much Google wants to crawl, based on your site's popularity and how often content changes). Together they set the practical ceiling on how much of your site Google visits.

Here is the reassuring honesty most SEO content leaves out: the vast majority of small local business sites never come close to their crawl budget. Google explicitly says crawl budget is generally not something owners of small or medium sites need to worry about. If your plumbing site has 40 pages, Google will happily crawl them all.

Crawl budget becomes a real concern only for large sites — think thousands or millions of URLs, like big e-commerce catalogs or listing sites. On those, wasting crawl budget genuinely matters:

• Endless parameter URLs (filters, sort orders) can eat crawl budget on near-duplicate pages.
• Broken links and redirect chains make Googlebot work harder for no reward.
• Low-value pages can crowd out important ones in the crawl queue.

If you run a small site, the better use of your energy is making sure pages get indexed and are worth ranking — not obsessing over crawl budget. If you run a large one, our technical SEO basics guide covers keeping Googlebot focused on what matters.