Soon after verifying your website to Google you can follow next by submitting robots.txt file at the root directory of your blog. This small text file is a command line to any search engine spiders which come to your website and tell them what and where to crawl pages on your website. Creating a robots.txt file is simply by making a .txt file with text editor such as Notepad and then type :

“User agent:*”
This command line will tell the spiders to crawl all the pages exist on your website. Then save this file with flename as robots.txt and then by using FTP sofrtware, upload it to the root directory of your website.

But if you want the search engine spiders do not crawl and index some certain files or folder on your website you can exclude them from spiders crawling by inserting command line such as :

“Disallow: /folder_name/” or “Disallow: /filename.html”
Creating a robots.txt file for your blog or website can be done through your Google Webmaster Tools account and then after uploading it to your website’s root directory, you can do a checking procedure whether your robots.txt file is working well or not. By submitting your robots.txt file will make your Busby SEO Test website or blog getting indexed more easly by Google search engine. For more details explanation about robots.txt file you can dig more deep on this Google guidelines.