Hi netweaver,
The robots.txt File
Search engines make use of a file called robots.txt to determine which web pages (and other files) are excluded from indexing when they visit your site. A sample robots.txt file has been included in your public_html directory.
Not all search robots honour this file. Never keep any sensitive information in your public_html directory as it could be viewed by anyone or indexed by search engine robots.
Some services are now indexing graphics on the web. If you don't want your images to be accessed in this manner, include them in your robots.txt file.
In case your robots.txt file is damaged or missing, here is a sample of the file. Simply cut and paste in-between the dashed lines.
Code:
---------
User-agent: *
Disallow: /images/
Disallow: /templates/
Disallow: /cgi-bin/
Disallow: *.jpg
Disallow: *.gif
---------
For more details try this link
http://www.robotstxt.org/wc/robots.html
Regards
jeet