SEO for Beginners
How Search Engines Work

Imagine the largest building in the world. Then imagine it filled - not with books, but loose pages of information. Billions of individual pages. Now imagine you need to know what type of fuel hose a TVR Chimera uses, or how to make Yorkshire Pudding when your mother-in-law is due to visit.

How do you find - amongst all those pages - not just any recipe or any fuel hose specification, but the exact ones you need? And quickly. VERY quickly.

This is the sort of problem search engines solve all day long and, given the amount of data they deal with every minute of the day, they're rather really good at it. So let's find out how search engines work.

1. Read and Index

This is the easy, but tedious bit of how search engines work. Automated programs called spiders 'crawl' pages of websites, reading the content. Spiders don't read as well as humans, so for the most part they guess what a page is about or look for clues on the page. Mainly, they scan the words used in the headlines, text, pictures and links to see if any are repeated more often than others.

The reasoning here is that if you write an article or blog post about David Coulthard, then his name is likely to show up more often than that of Lewis Hamilton. So the spider would - hopefully - conclude that the page is about David Coulthard.

Since you'd also be using words like F1, Lotus, McLaren, Grand Prix etc. the spider would eventually file that page under 'motorsport' and maybe also under 'celebrities' and a number of other topics.

That's the indexing part of how search engines work. Placing each page that has been read into a giant database of various topics. Without being in the index, your site will never be found - simply because the search engines don't know about it.

Most people want their pages to be found. But if you'd rather keep some pages to yourself, then there's even a special command called 'noindex' that you can use to tell the spiders not to include that page in the database.

2. Search

Here, customers and search engines come together. You need a new fuel hose for your TVR Chimera. But you've mislaid the handbook and you have no idea which type to order. You access a search engine and type fuel hose TVR Chimera.

And you expect to see a page of information about TVR fuel hoses, including images, links to merchants, fitting instructions, other drivers' experiences with each product and advertisements for specialists offering to fit them for you.

To be able to deliver this information, the search engine needs a) the database it created earlier and b) the index it also created earlier. These two will deliver a list of all pages that have the words fuel hose and TVR Chimera on it.

There will still be millions of pages left, and you can't be expected to read them all. So next, our search engine needs a brain. That brain is commonly called an algorithm - a bit of maths that sorts through all the initial results and discards any that are not relevant, like pages about fuel hoses for TVR Griffiths, or Facebook comments about TVR, or fuel hoses, or maybe even pages full of unrelated special offers.

It does this over and over - until it has only the most relevant pages of information left. And as this will still be hundreds or even thousands of pages, the search engine goes another step further.

3. Ranking

This is the really clever bit of how search engines work - and the one most SEOs try to influence with their techniques. Using yet another algorithm, the search engine now grades or 'ranks' the remaining pages in order of importance, offering you the most relevant, most important ones first.

If it succeeds, you're one happy camper. You know the exact type of fuel hose you need, you have fitting instructions, you even found a guy down the road who specialises in replacing TVR Chimera fuel hoses. Excellent.

So how does a search engine measure importance? How does it decide which page deserves to be top, which sites should be listed on the first page of results and which sites should end up on page 57? Now that you know how search engines work, you can lean about ranking in part 4 of SEO for Beginners.

Am I No 1? How search engines rank their results.