Thread: what is a robot.txt file
View Single Post
  #7 (permalink)  
Old 08-12-2009, 01:01 PM
mark spenser mark spenser is offline
Junior Member
  
Join Date: Jun 2009
Location: Los Angeles, CA
Posts: 22
mark spenser is on a distinguished road
Thumbs up it is guide to search engine crawlers
it commands search engine crawlers that which pages to crawl and which pages not.
Reply With Quote