Please Read First!: What is robots.txt for?

started by Raghav Raaz on Jan 4, 2014 — RSS Feed

Raghav Raaz Raghav Raaz
Posts: 122

Hello all
What is robots.txt file and what use in this please give our suggestion.



Thanks
Raaz
SEO India Local SEO Services Organic SEO Services

Aredey Axh Aredey Axh
Posts: 775

A simple text file that stops Google (and other search engines that recognize the file and its commands) from crawling the site, selected pages in the site, or selected file types in the site.

Batterie Acer
Batterie Apple
Batterie Asus
Batterie Clevo
Batterie Dell
Batterie Fujitsu
Batterie Gateway
Batterie HP
Batterie Lenovo
Batterie LG
Batterie Medion
Batterie Motion
Batterie MSI
Batterie Packard Bell
Batterie Panasonic
Batterie Samsung
Batterie Sony
Batterie Toshiba
Batterie Vizio
Chargeur Acer
Chargeur Apple
Chargeur Asus
Chargeur Braun
Chargeur Clevo
Chargeur Dell
Chargeur FSP
Chargeur Gateway
Chargeur HP
Chargeur Lenovo
Chargeur LG
Chargeur MSI
Chargeur Packard Bell
Chargeur Panasonic
Chargeur Philips
Chargeur Samsung
Chargeur Sony
Chargeur Toshiba
Acer AC Adapter
Apple AC Adapter
Asus AC Adapter
Braun AC Adapter
Clevo AC Adapter
Dell AC Adapter
eMachines AC Adapter
FSP AC Adapter
Fujitsu AC Adapter
Gateway AC Adapter
HP AC Adapter
Lenovo AC Adapter
LG AC Adapter
Medion AC Adapter
MSI AC Adapter
Packard Bell AC Adapter
Panasonic AC Adapter
Samsung AC Adapter
Sony AC Adapter
Toshiba AC Adapter
Vizio AC Adapter
Acer Battery
Apple Battery
Asus Battery
Clevo Battery
Dell Battery
Fujitsu Battery
Gateway Battery
Gigabyte Battery
HP Battery
Lenovo Battery
LG Battery
Medion Battery
MSI Battery
Packard Bell Battery
Panasonic Battery
Samsung Battery
Schenker Battery
Sony Battery
Toshiba Battery
adattatore alimentatore

http://www.lcddisplayscreens.co.uk/

Sam Austin Sam Austin
Posts: 5

A file written and stored in the root directory of a website that restricts the Search Engine spiders from indexing certain pages of the website.

Aredey Axh Aredey Axh
Posts: 775

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site (i.e. it is not a firewall, or a kind of password protection) and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.
Adaptador Ordenador Portátil
adaptateur ac
ac adapter
adattatore alimentatore

http://www.lcddisplayscreens.co.uk/

Aredey Axh Aredey Axh
Posts: 775

chargeur acer iconia tab a700
chargeur acer iconia tab a510
chargeur acer iconia tab a500
chargeur asus eee pc 1015b
vgp ac10v10
MacBook Pro 13
chargeur asus eee pc 1001pxd
chargeur asus exa1004eh
HP Officejet Pro 8500
chargeur dell vostro 3700
chargeur ordinateur samsung NP550P7C
chargeur Asus EXA0904YH
chargeur Asus ADP-90CD DB
chargeur ASUS X550CA-XX198H
chargeur Asus ADP-180HB
asus zenbook ux31e chargeur
chargeur Asus ADP-90YD B
chargeur asus PA3715U-1ACA
batterie samsung r730
chargeur toshiba pa3917u 1aca
chargeur HP Pavilion Sleekbook 15-b042sf
chargeur HP Pavilion Sleekbook 15-b044sf
chargeur Acer GFP121T-1215 ADP-18TB A
chargeur Acer ADP-65VH B
Samsung PA-1400-24 chargeur
chargeur Sony S1 Tablet SGPT111
chargeur VivoBook X201E-KX097H
chargeur Sony SGPAC10V1
chargeur HP t5740
batterie toshiba PA5109U-1BRS
chargeur portable 8470p hp

http://www.lcddisplayscreens.co.uk/

Say My Vows Say My Vows
Posts: 47

Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.
Vera Wang Wedding Dresses

Say My Vows

Corporate Supp Corporate Supp
Posts: 60

Very knowledgeable thread.

Recruitment Consultants India, HR Business Process Consulting Solutions

You must login to reply