How Do Search Engines Work?

SEO can be defined as the activity of optimizing Web pages or whole sites in order to make them more search engine-friendly, thus getting higher positions in search results. One of the facts about SEO is that, no matter how much optimization and work you do on your site to make it to the top, if you miss the basic things or do it wrong way, all your efforts will go in vain. So be careful to get the basics right! Also try to set some realistic goals for yourself - like getting into top 30 for a certain keyword of yours rather than in top 3 or top 10. This way you won't get upset and also will be motivated to work more.

So how Search Engines work actually? As you might know that SE's (Search Engines) are not human but bots. (Bot is a program designed to do specific tasks). Since they are not human, it's obvious that they way they view a web page is different than we humans view it. So it becomes necessary to understand as to how these bots work so that we can optimize our site to the fullest. Unlike human beings, bots cannot see banners, images, audio/video. They can only view the text. Hence SE's are text driven.

Search Engines send bots (also know as crawler or spider) to crawl your web page. Now these crawlers follow one link from another and index everything that comes. However it's obvious that these crawlers or spiders cannot crawl each and every website everyday due to their vast numbers. So they crawl occasionally and AFTER this your new post/information if indexed/updated in SE. If the crawler doesn't visit your site even after you make any changes to it or make a post, your changes won't be reflected in SE's until your site is crawled.

Now after the page is crawled, it is then indexed and stored in a large database. From here SE's rank/list your site for relevant search terms. Now this priority is decided by Page Rank also know as PR as well as your keywords and number of pages indexed. We will discuss each of them later. So your goal should be to get maximum of your pages crawled so that your site get's indexed faster. You can check your crawl rate through various online tools available. just Google for "Crawler checker" and you should be able to get many results. Also you can increase this crawling rate from your Google Webmaster's account. Here's how you can do it:

1. Log in to your Google Webmaster Tools account (Sign up here if you haven't already!)

2. Add your site to the Webmater.

3. Complete the verification of your site by following relevant steps as mentioned.

4. If you have finished the first three steps already, then just click on 'Settings' link on the left hand side of your control panel.

5. There you should be able to see 'Set Crawl Speed' option. Check it to maximum and this will reflect in about a month or so.

Note: Setting crawl speed might send you large number of crawlers and thereby increase your server load.