SeriesComing Soon

Crawling

How search engine crawlers discover, request, and process web content.

1 articles in this series

Crawling

Test the page Google can actually process

A page can look correct in a browser while serving incomplete HTML, blocked resources, or different content to a crawler. Start with the guide to viewing a site as Googlebot for a practical workflow using Search Console, rendered HTML, and browser developer tools.

Crawling and rendering happen before indexing, so connect the results to the web indexing process. The broader search engine basics guide explains where discovery, fetching, rendering, indexing, and ranking fit together.

Future supporting guides will cover detailed Googlebot views and the tools formerly known as Google Webmaster Tools. Use this hub to move from a visual symptom to the correct crawl or rendering diagnosis.