Surpass Web Hosting Forums - View Single Post

Giles · June 30th, 2009, 11:52 PM

Probably the best thing to do is create a robots.txt file. These are the first thing that search engine spiders look for and from the code in that and from there index your site.

You can find loads of information from google on this.
robots.txt - Google Search

robots.txt generator
Robots.txt Generator - McAnerin International Inc.

June 30th, 2009, 11:52 PM	#2 (permalink)
Giles Rolling with the punches Comfy Contributor Joined in Oct 2008 Lives in a constant state of confusion thanks to all of this buffonery caused by Twitter and other social networking sites that bring us further and further apart from each other by giving us a chair to sit in instead of exploring the real world with friends. Hosted on CocaCola and Pop Rocks. 254 posts Gave thanks: 32 Thanked 15 times	Probably the best thing to do is create a robots.txt file. These are the first thing that search engine spiders look for and from the code in that and from there index your site. You can find loads of information from google on this. robots.txt - Google Search robots.txt generator Robots.txt Generator - McAnerin International Inc.