View Single Post
  #2 (permalink)  
Old 05-07-2005, 03:25 PM
jitender's Avatar
jitender jitender is offline
Site Admin
 
Join Date: Jun 2004
Posts: 1,200
jitender has a spectacular aura aboutjitender has a spectacular aura aboutjitender has a spectacular aura about
iTrader: (0)
Default

Dear Vineet,

Robots.txt file used for search engine robots,

it's direct to robot where they go, access area or non access area,

if you want to disallow to robot to read a cgi-bin directory write below

Code:
User-agent: * Disallow: /cgi-bin/

or you want to stop google robot to read you site's particular area use below

Code:
User-agent: googlebot Disallow: /cgi-bin/

And if you want to stop read a particular file

Code:
Disallow: email.htm

I think it will help you

Regards
Jitender
__________________
Free Webmaster Tools - Free SEO Tools
phpLDHacks - Add your template or Hack and earn money
Text Link Ads - Buy & Sell text link ads
Reply With Quote