How SEO Works in 2026: Crawling, Rendering, Indexing, Ranking and AI Overviews, Step by Step

SEO works by getting a page through a chain of machine steps: a search engine discovers the URL, crawls it, renders it, indexes it under one canonical address, ranks it against every other page for a query, then builds a results page that may open with an AI Overview. Each step can fail independently, and a B2B site controls more of that chain than most teams assume.

This is part one of our series on how search and AI visibility work. It explains classic search as it runs in 2026, because every other discipline in the series sits on top of it. If you want the definitions first, read SEO vs AEO vs GEO explained. If you already know the basics and want the answer layer, skip to how AEO works. The mechanics below come from Google's own documentation, linked inline, because Google is the system most B2B buyers still start with and the one whose documentation is most complete.

The short version

1. Discover Google finds your URL through links on pages it already knows and through your sitemap.
2. Crawl Googlebot requests the page, if robots.txt and your server allow it, and downloads the HTML.
3. Render A headless Chromium browser runs your JavaScript so Google sees the page a visitor sees.
4. Index Google analyzes the content, groups duplicates, picks a canonical URL and stores the page.
5. Rank For each query, ranking systems order indexed pages by relevance, quality, links and context.
6. Serve Google assembles the results page: links, snippets, rich results and often an AI Overview.

How a page gets from your server to a search result, step by step

Google's in-depth guide to how Search works names three stages: crawling, indexing and serving. The seven steps below split those stages into the parts you can actually diagnose and fix.

Step 1: Discovery

Before Google can crawl a page, it has to know the URL exists. There is no central registry of web pages. Google finds new URLs mainly by extracting links from pages it has already crawled and by reading sitemaps that site owners submit. A page nobody links to, sometimes called an orphan page, may never be found.

Links only count for discovery when they are real links. Google's JavaScript SEO basics say it extracts links from <a> elements with an href attribute. A menu that navigates with click handlers, or a resource library that loads items only after a filter is clicked, can hide pages from discovery even though visitors reach them easily.

B2B sites create orphans all the time: campaign landing pages linked only from ads, case studies reachable only through a filtered grid, and integration pages that exist in the sitemap but nowhere in the navigation.

Step 2: Crawling

Crawling is Googlebot requesting the URL and downloading what the server returns. Google decides algorithmically which sites to crawl, how often and how many pages to fetch, and it slows down when a server responds with errors so it does not overload it. Most requests come from the smartphone version of Googlebot, because Google indexes the mobile version of content first.

Three details matter for B2B teams. First, robots.txt controls crawling, not indexing. Google's robots.txt introduction warns that a disallowed URL can still be indexed if other sites link to it; to keep a page out of results, use noindex or password protection. Second, there is a size ceiling: Googlebot downloads the first 2MB of an HTML or supported text file (64MB for PDFs), and CSS and JavaScript files are fetched separately under the same limit. Third, a firewall or bot-protection rule that challenges Googlebot stops the chain here, however good the page is.

Step 3: Rendering

Many sites send a thin HTML shell and build the visible page with JavaScript. Google handles that by rendering. According to its JavaScript documentation, every page that returns an HTTP 200 status goes into a render queue, where a headless Chromium browser executes the scripts. That can happen seconds after crawling or take longer, depending on resources. Google then indexes the rendered HTML.

Two traps are worth knowing. If the initial HTML carries a noindex tag, Google may skip rendering entirely, so JavaScript that later removes the tag will not help. And Google states plainly that not all bots can run JavaScript. That matters more every year, because most AI crawlers read the raw HTML only, which is covered in how AI crawlers work. Server-side rendering or prerendering removes the risk for every reader at once.

Step 4: Indexing

Indexing is where Google works out what the page is and whether to store it. It analyzes text, title elements, headings, image alt attributes and structured data. It then runs canonicalization: it groups pages with the same or very similar content into a cluster and picks the most representative URL as the canonical, the one that can appear in results. Signals collected here include the page's language, the country it targets and its usability.

Google's technical requirements set the minimum for eligibility: Googlebot is not blocked, the page returns HTTP 200, and it has indexable text that does not violate spam policies. Meeting them makes a page eligible. Google is explicit that indexing is not guaranteed. Low-quality or near-duplicate pages are often crawled and then left out.

The URL Inspection tool in Search Console shows the result of this step for any URL: whether it is on Google, whether crawling and indexing were allowed, the canonical Google selected (which may differ from the one you declared), and the HTML Google actually crawled.

Step 5: Ranking

When someone searches, Google does not search the web. It searches its index, then orders candidate pages. Its guide to ranking systems lists the named systems involved, which fall into a few groups:

• Understanding the query. BERT, RankBrain and neural matching relate words to concepts, so a page can match a query it never repeats word for word.
• Relevance at page and passage level. The passage ranking system identifies individual sections of a page to judge how relevant the page is to a search. A long guide can rank for a narrow question answered well in one section.
• Quality and reliability. Reliable information systems, the reviews system and original content systems favor authoritative, first-hand material. Google's helpful content guidance says E-E-A-T is not a specific ranking factor, but its systems look for signals of experience, expertise, authoritativeness and trust, and trust matters most.
• Links. Link analysis systems, including PageRank, use how pages link to each other to understand what they are about.
• Freshness and variety. Freshness systems surface newer content where recency matters. Deduplication removes near-identical results, and the site diversity system generally shows no more than two listings from the same site.
• Spam. SpamBrain and related systems demote or remove pages that violate spam policies.

Context then adjusts the order: Google's overview names the searcher's location, language and device as factors. It also states that it does not accept payment to rank pages higher. Ads are sold separately and labeled.

Step 6: Results page assembly

The results page is built from components, which Google documents in its visual elements gallery. A standard text result has a title link, a snippet, attribution (favicon, site name, URL or breadcrumb) and sometimes sitelinks. Rich results typically rely on structured data. Related questions groups and related searches appear around them.

Featured snippets show a page's text above its link. Google says its automated systems decide whether a page makes a good featured snippet and that you cannot mark a page up to become one. You can opt out with nosnippet, reduce eligibility with max-snippet, or exclude specific text with data-nosnippet.

Step 7: AI Overviews on top

For many queries, the results page now opens with an AI Overview, and a separate AI Mode offers a conversational version. Google's guide to generative AI features in Search explains that these features ground their answers by using the core Search ranking systems to retrieve relevant pages from the Search index, and use query fan-out, generating related queries to fetch more results. Google's AI features documentation sets the eligibility bar: a page must be indexed and eligible to show with a snippet. There is no separate AI index to submit to.

That is why steps 1 to 6 still decide who gets cited. It also changes what a click is worth. Seer Interactive's 2026 study of 5.47 million queries found AI Overviews on 36 percent of informational queries, 95.4 percent of comparison queries and 85.9 percent of question-format queries, which describes most B2B research. In Search Console, Google counts an AI Overview as a single position, with every link inside it sharing that position. The full mechanism is in how AI Overviews work.

What happens at each stage, and how B2B sites break it

What you control: a B2B checklist by stage

• Discovery. Every commercial page is linked from at least one crawlable HTML link, and the XML sitemap lists only canonical, indexable URLs.
• Crawl access. robots.txt allows Googlebot on all public pages, and your CDN or web application firewall does not challenge verified search crawlers.
• Server health. Key pages return HTTP 200, redirects are single hops, and retired pages return 404 or 410 rather than a soft "not found" page.
• Rendering. Main copy, headings and links appear in the raw HTML through server-side rendering or prerendering. No noindex in the initial HTML of pages you want indexed.
• Canonicals. Each piece of content has one URL. Tracking parameters, filtered views and print versions point a canonical at it.
• Content quality. Pages show who wrote them, draw on first-hand experience with named customers, numbers and screenshots, and answer the question a buyer typed.
• Internal links. Pillar pages link to the detail pages beneath them, using descriptive anchor text instead of "learn more".
• Titles and snippets. Title elements name the topic and the company; opening paragraphs work as a snippet on their own.
• Structured data. Organization, Article and Breadcrumb markup match visible content. See how entities and schema work.
• Reputation. Earned mentions and links from industry publications, partner directories and review platforms, which feed link analysis and corroborate claims.
• Measurement. Search Console verified for every property, with monthly exports of queries and pages, and Bing Webmaster Tools set up alongside it.

Our AEO checklist for a B2B website turns the access and entity items into pass or fail tests.

Illustrative example: why a compliance software page never ranked

Illustrative example. The company, pages and findings below are invented to show the diagnostic sequence. They are not a client result.

A 140-page B2B software company sells compliance automation to fintech firms. Its most important page, "SOC 2 automation for fintech", has been live for a year and does not appear in the top 50 results for any query containing "SOC 2 fintech". The team's instinct is to rewrite the copy. Working the chain in order finds the real problems first:

1. Discovery. The page is in the sitemap, but the only link to it sits in a JavaScript mega menu that opens on hover. A crawl finds no <a href> pointing to it. Fix: add plain HTML links from the product page and the fintech industry page.
2. Crawling. Server logs show Googlebot fetching the page every few weeks with HTTP 200. Nothing to fix.
3. Rendering. View Source shows a headline and an empty container. The body copy, the control-mapping table and the customer quotes load from a headless CMS through client-side JavaScript. URL Inspection shows Google renders most of it, but AI crawlers would see almost nothing. Fix: server-side render the page.
4. Indexing. URL Inspection reports the Google-selected canonical is a different URL: a copy with ?utm_source=partner that a partner linked to heavily. Signals are split across two addresses. Fix: add a self-referencing canonical and ask the partner to update the link.
5. Ranking. Only now does content matter. The top results name specific controls, audit timelines and fintech regulators. The page says "streamline compliance". Fix: add a section per control family, a named customer example, the author's audit credentials and an update date.
6. Results page and AI Overview. The query "how long does SOC 2 take for a fintech startup" shows an AI Overview citing two competitors. Fix: open the relevant section with a two-sentence direct answer, then the detail.

The order matters. Rewriting copy first would have changed nothing, because Google was reading a different canonical and AI crawlers were reading an empty page. Real outcomes from this kind of work vary by market; for a documented one, see how a rebuilt site and content program took HiddenBrains from 50,000 to 600,000 monthly users, or how organic search built a $6.4M pipeline for IQLECT.

Four misconceptions that waste B2B budgets

• "robots.txt keeps a page private." It stops crawling, not indexing, and it is not access control. Use noindex or authentication.
• "Schema makes pages rank." Google uses structured data to understand pages and enable rich results. Its structured data introduction says markup does not guarantee rich results, and it never describes markup as a ranking boost.
• "AI Overviews need a separate optimization track." Google says there are no additional requirements beyond being indexed and snippet-eligible. The work is the same chain done well, plus pages that answer questions directly.
• "More pages means more traffic." Near-duplicate industry or location pages get clustered under one canonical, or left out of the index. Our note on scaled content abuse covers the downside.

Glossary

• Canonical URL: the one URL Google selects to represent a cluster of duplicate or near-duplicate pages.
• Crawl: an automated request that downloads a page so a search engine can process it.
• Googlebot: Google's main crawler for Search, running as a smartphone and a desktop variant.
• Index: Google's stored, analyzed copy of the pages it may show in results.
• Passage ranking: a Google system that judges relevance using individual sections of a page.
• Render queue: the backlog of crawled pages waiting for Google to execute their JavaScript.
• robots.txt: a file at the site root telling crawlers which URLs they may request.
• SERP: search engine results page, the assembled page of links, features and AI answers.
• Snippet eligibility: whether a page's text may be shown as a preview, which AI Overviews also require.
• Structured data: machine-readable markup, usually JSON-LD, describing what a page is about.

Next in the series

Classic search decides whether your pages are in the pool. The next explainer, how AEO works, covers what happens when an engine reads those pages and writes an answer instead of a list. For the full method with a self-audit and a 90-day plan, get the B2B AEO and GEO playbook, or ask for a free audit of where your site breaks the chain.