I use search engines every day and had only a vague idea how they actually rank results, so I built one — over the real content of this site — to find out. It lives in the portfolio as the Search Engine demo.
The core idea is an inverted index: instead of scanning every page for a query at search time, you scan every page once, ahead of time, and record which words appear where. A query then becomes a fast lookup instead of a full re-read. The harder part is ranking — matching isn't enough, you need to know which matches matter. That's where BM25 comes in: a formula that scores a page higher when a term is rare across the whole site but frequent on that specific page, with diminishing returns so a page can't win just by repeating a word.
Typo tolerance was the part I underestimated. "Correct spelling only" search feels broken the moment a real person types into it, so the index also matches near-misses — close enough in edit distance to count, not so close that unrelated words start colliding.
What stuck with me is how much of "the algorithm" is really "the scoring function." The index is mechanical; the judgment call is entirely in how you weigh rarity against frequency against length. Small changes to that formula visibly reshape which results feel right — which is probably why real search engines treat their ranking as the actual product.