what is ROBOT .txt file value?

[ravirms]

I have still confusion in ROBOT ,only i heard abt allow or disallow of robot command.
as i know ROBOT is used for hide or unhide the text.

I have lots of confusion about ROBOT and how it works plz
help for my opinion to all readers

[bermuda]

If you are going to give search engines permission to access all folders and parts of your website, you may even completely forget about uploading a text file called the Robots item on your server because the instructions added to the files are in most cases intended to block some folders or URLs from getting accessed.

[ndrewstrauss]

It is a text and it should not be mistaken for a HTML file. It basically instructs search engines directing them to visit some pages and avoid visiting some pages.

[spinxwebdesign]

Most sites contain pages that should not be indexed by the search engines. Administrative pages, for example, ABC's "contact" page: "contact.html". There's no need to have it indexed, so we use the robots.txt file to tell the search engine spider (robot) to ignore it.

[ndrewstrauss]

The Robots.txt is a easy text file so as to tells the search engine not to throat or indexes a exacting web page, file or directory. It is stored in a website.

[eoinbronks]

Robots.txt file is presently an effortless text file stored in the web server's root directory. It confines the web spiders while indexing the web page..

[tanja]

It's a file that can ban crawlers from searching and examining your website, e.g not indexing some pages .This file can be created with a notepad

[Alice]

its a text file to show the crawler to index or don't want

[SexyTracey]

A well written robots.txt file helps improve search engine rankings by providing important information to the search engine bot.
When a search engine crawler comes to your site, it will look for a special file on your site. That file is called robots.txt and it tells the search engine spider, which Web pages of your site should be indexed and which Web pages should be ignored.

[carriethomson]

Originally Posted by SexyTracey

A well written robots.txt file helps improve search engine rankings by providing important information to the search engine bot.
When a search engine crawler comes to your site, it will look for a special file on your site. That file is called robots.txt and it tells the search engine spider, which Web pages of your site should be indexed and which Web pages should be ignored.

I agree with you. Robot.txt tells search engine which files to crawl and which not to crawl.

[stevemack]

it is basically a text file, which tells search engines what to do on the website. Just passing instructions/operating search engines with this text file. Before crawling your website, bot looks for Robot.txt file only first.

[vstrategy]

In SEO the ROBOT are called as crawlers. They crawl upon your page in the certain site to know by search engines if your keyword is appropriate to your site.

[Sabrina178]

It's there for you to tell the spiders which pages of your site you *don't* want indexed.That saves bandwidth. Lack of a robots.txt file will mean the spiders will try and index everything.

[peterneeson1]

It usually incorporates the pages that are yet to be indexed by search engine. It will restrict the search engine from accessing certain pages of your website.

[eyanharve]

robots.txt file using for not allowing specific web page and website content to search engine.

[carriethomson]

I think robots.txt file is used to tell search engines which files to crawl and which not.

[robinjackman]

robots.txt - the one file which indicate about sitemap to search engine and also it helps for allow, disallow any webpage for search engine.

[zackdsouza1]

Robot.txt is text file that is important if you want to keep your private pages private and do not want to publish it to viewers.

[crescendo]

Originally Posted by bermuda

If you are going to give search engines permission to access all folders and parts of your website, you may even completely forget about uploading a text file called the Robots item on your server because the instructions added to the files are in most cases intended to block some folders or URLs from getting accessed.

Yes, this is right information for robot.txt file

[freeshipping]

robots.txt - the file you can tell the serch engines to access and serch which folder of your website,and it is very helpful,so it is also very important too.