A Dangerous Search Engine Optimization Mistake.
Submitted By: Julia Udovenko
Wordcount: 572


href= coupondirectory.net website that caused me so much grief showed that Google crawled many of its pages, but each and every one of them showed some robots.txt unavailable error. Reading on, I discovered that Google did crawl my coupondirectory.net website, in fact it crawled it many times, but before indexing any of my pages, it looked for the robots.txt file, but since I didnt have one, it couldnt access it. Also having received a 200-OK code instead of 404-not found, it assumed that robots.txt exists but is presently unavailable and therefore, rescheduled the crawl for a later time. In my case its been rescheduling it for a few months while I was desperately trying to promote my coupondirectory.net website.

Imagine my reaction when I found out this. The evi.combination of always returning code 200-OK and absence of the robots.txt is deadly for search engine optimization. So if you are experiencing a similar situation, ask your web hosting provider whether they return any HTTP error codes, 404 in particular, and make sure you have the robots.txt file in your root directory.



"

Article Source: http://www.work-at-home-business-website.com/article/34036