How to Add Custom Robots.txt file in Blogger

What is Robots.txt?

Robots.txt is a text file which contains few lines of simple code. It is saved on the website or blog’s server which instruct the web crawlers to how to index and crawl your blog in the search results. That means you can restrict any web page on your blog from web crawlers so that it can’t get indexed in search engines like your blog labels page, your demo page or any other pages that are not as important to get indexed. Always remember that search crawlers scan the robots.txt file before crawling any web page.

Each blog hosted on blogger have its default robots.txt file which is something look like this:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED

Explanation

This code is divided into three sections. Let’s first study each of them after that we will learn how to add custom robots.txt file in blogspot blogs.

How to block a web site in Windows 7

How to block a web site in Windows 7
Windows operating system is having some many features that are not visible to all. One such feature is blocking the web sites which we don't want to see. You children may browser your system and you want to block some known sites like games, other adult sites. In Windows Vista and Windows 7 operating system you can block the sites without any 3rd party software.

All you need to do is to add the web sites in hosts file which is in Windows system folder. This is not a new concept which exists in old version of Windows operating system. It requires administrator permission to modify and save the hosts file.

Host file will map the IP address to a host and it is controlled by the administrator of the system.

To block Web sites

Open the location C:\Windows\System32\drivers\etc and you can see a list of files.

Open the hosts file in a notepad and you can a default address with 126.0.0.1 and localhost. Below that add the web sites that you want to block. One web site can be added in one line (check the below screen shot) Enter the Default IP address and the website address name

All the IP address should be 127.0.0.1 and the web sites can be any web sites that you want to block. Once you added all the web sites save the file. Even as a administrator you cannot edit the file. You need to assign permission to edit the file and you can start editing the file. Any time you can edit the hosts file and you can add or remove the web sites to allow or block.