what is robots.txt

[shengmaoda]

The robots.txt is a stored in the root directory of the web site under ASCII coding text files, it usually against Internet search led also called web spiders, which of the site should not be content is search engine for the rovers, which can be obtained

[Motorcyclestore]

Robots.txt is a file which tell search engine crawlers that which pages should be crawled and which should not be.

[kradha707]

This file is like primary information to the spiders, that which pages that spiders should not be crawled or indexed. If you have a sensitive information like account or login information etc...

[larry05]

Robots.txt is a file which helps search engines know which pages to crawl and which not to do. If there is some private information which you dont want to display in search engines then simply writing disallow command in robots file will resolve the issue.

[traveloworld]

Hi,
Robot.txt is the process where we hide some page which we don't want to cache by any search engine.

[menelectric]

Hey there robot.txt is a file that can allow search engines to crawl certain pages and have nofollow for others.

Example, if you have a project your working on and dont want it to be visible to the search engines you'd do a nofollow robot.txt

[punitbook]

robots.txt is a controller of search engine that give instruction to search engine crawler that what to crawl and what not to be. It’s not compulsory but really important part of SEO

[archives]

Robert.txt is a file which indicate Search Engine that which pages spiders should crawl and which not to be. if we want to take hide any information like any page or login information any thing Robert.txt will be hide it.

[MichaelJames]

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do

[seomaartin]

Robots.txt crawl the website page easily that is beneficially to the indexing the website on search engine easily.

[samualsmith45]

Robots.txt file is for crawler to know which information of the web page is not to index. Robots.txt is very important from seo prospective because whenever crawler searching for any keyword first thing crawler check for robots.txt and sitemap.

[affortableseo]

Robots.txt is the file where we have to give instructions to search engine crawlers.

[bonzoi]

Robots.txt is a file that you put in to your website that instruct the search engines robots that which page you would like to visit them and not to visit them.

[daddyclassifieds]

Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.

Google specifically follows instructions given in this robots.txt file

[jewelista]

Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders. Web authors can use robots.txt to keep cooperating Web robots from accessing all or parts of a Web site that you want to keep private.

[myadsbook]

this is a text file on root directory. you can edit this and control the search engine crawler

[deepak1]

hi,
Robots matatag can simply go unnoticed. A better way to inform search engines about your will is to use a robots.txt file for Your Website

[albertsmith]

Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website. Google specifically follows instructions given in this robots.txt file.

[rajb]

Robots.txt file is a set of instructions that tell search engine robots which pages of your site to be crawled and indexed. In most cases, your site is consist of many files or folders i.e. admin folders, cgi-bin, image folder, which are not relevant to the search engines. Robots.txt helps tell spiders what is useful and public for sharing in the search engine indexes and what is not.

[ymagestore]

Robots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders.