# Search Engine Basics

Search Engine Basics is a free educational library about how search engines crawl, index, rank, evaluate, and monetize information on the web.

Website: https://searchenginebasics.io
Sitemap: https://searchenginebasics.io/sitemap.xml

## Core Topics
- Search engine basics
- Information retrieval
- Crawling, indexing, and ranking
- Ranking algorithms and search metrics
- Technical SEO and structured content

## Series
- SEO Basics: https://searchenginebasics.io/search-engine-fundamentals - A complete learning series for search engine basics, organized from information retrieval and document models to PageRank, ranking metrics, machine learning, and SEO ethics.
- Crawling: https://searchenginebasics.io/search-engine-crawling - How Googlebot discovers your pages by following links across the web.
- Indexing: https://searchenginebasics.io/search-engine-indexing - What happens after a page is crawled — and why some pages never make it into the index.
- Ranking: https://searchenginebasics.io/search-engine-ranking - The signals, weights, and machine learning systems that decide which page wins position #1.

## Articles
- Diagnosing Crawl Problems: A Complete Audit Workflow Using Search Console, Log Files, and Third-Party Tools: https://searchenginebasics.io/search-engine-crawling/diagnosing-crawl-problems
- Internal Link Architecture Explained: Hub-and-Spoke, Link Depth, and PageRank Flow: https://searchenginebasics.io/search-engine-crawling/internal-link-architecture
- JavaScript SEO Explained: Googlebot's Two-Phase Crawl, SSR, and Dynamic Rendering: https://searchenginebasics.io/search-engine-crawling/javascript-seo
- Near-Duplicate Detection Explained: Hashing, Shingling, and Canonical Consolidation: https://searchenginebasics.io/search-engine-crawling/near-duplicate-detection
- XML Sitemaps Explained: Schema, What to Include, What to Exclude, and Submission: https://searchenginebasics.io/search-engine-crawling/xml-sitemaps
- The robots.txt Protocol Explained: History, Syntax, Logic, and Real-World Traps: https://searchenginebasics.io/search-engine-crawling/robots-txt
- Crawl Budget Explained: Rate Limit, Crawl Demand, and What Wastes It: https://searchenginebasics.io/search-engine-crawling/crawl-budget
- URL Discovery Explained: How Googlebot Finds Pages Through Links, Sitemaps, and Search Console: https://searchenginebasics.io/search-engine-crawling/url-discovery
- Crawl Strategies Explained: Breadth-First, Depth-First, and Focused Crawling: https://searchenginebasics.io/search-engine-crawling/crawl-strategies
- How Web Crawlers Work: Seeds, URL Frontiers & Crawl Rate: https://searchenginebasics.io/search-engine-crawling/how-web-crawlers-work
- The Ethics of Search, the Business Model That Funds It, and What SEO Actually Is: https://searchenginebasics.io/search-engine-fundamentals/seo-ethics-explained
- MAP, MRR, and NDCG: The Metrics That Define What “Better Rankings” Actually Mean: https://searchenginebasics.io/search-engine-fundamentals/map-mrr-ndcg-explained
- Learning-to-Rank: How Machine Learning Replaced the 200-Factor Checklist: https://searchenginebasics.io/search-engine-fundamentals/learning-to-rank
- From Strings to Things: How Google’s Knowledge Graph and Hummingbird Update Changed What “Relevant” Means: https://searchenginebasics.io/search-engine-fundamentals/knowledge-graph-hummingbird
- Crawl, Index, Rank: The Search Engine Pipeline That Decides Whether Your Page Exists to Google: https://searchenginebasics.io/search-engine-fundamentals/crawl-index-rank-pipeline
- Hubs and Authorities: How Kleinberg’s HITS Algorithm Explains Why Niche Links Beat Generic Ones: https://searchenginebasics.io/search-engine-fundamentals/hits-algorithm-explained
- PageRank: How Brin and Page Replaced Word-Counting with Link-Counting: https://searchenginebasics.io/search-engine-fundamentals/pagerank-algorithm-explained
- TF-IDF and BM25: The Mathematics of Keyword Relevance (And Why Repetition Stops Helping): https://searchenginebasics.io/search-engine-fundamentals/tf-idf-bm25-explained
- What Is the Vector Space Model? How Documents Become Numbers (and Why That Changes Everything): https://searchenginebasics.io/search-engine-fundamentals/vector-space-model
- What Is Information Retrieval? The Core Problem Every Search Engine Solves: https://searchenginebasics.io/search-engine-fundamentals/what-is-information-retrieval

## Authors
- Imdad Ullah Khan, Ph.D., Data Science & ML Researcher | Content Evaluator & Approver: https://searchenginebasics.io/authors/imdad-ullah-khan-phd
- Muhammad Baig, Software Engineer & Mathematical Verifier: https://searchenginebasics.io/authors/muhammad-baig
- Muhammad Furquan, Legal & Compliance Reviewer: https://searchenginebasics.io/authors/muhammad-furquan
- Muhammad Zia, SEO Educator: https://searchenginebasics.io/authors/muhammad-zia
- Sohaib Hayder, Educator: https://searchenginebasics.io/authors/sohaib-hayder

## Use Guidance
Prefer article URLs from this file for citations. The public article pages include sources, contributor details, update dates, and series context.