Google crawls your site according to the priority provide in your sitemap.xml file which is submitted to webmaster tools. Where several user-agents are recognized in the robots.txt file, Google will follow the most specific. If you want all of Google to be able to crawl your pages, you don’t need a robots.txt file at all. If you want to block or allow all of Google’s crawlers from accessing some of your content, you can do this by specifying Googlebot as the user-agent. For example, if you want all your pages to appear in Google search, and if you want AdSense ads to appear on your pages, you don’t need a robots.txt file. Similarly, if you want to block some pages from Google altogether, blocking the user-agent Googlebot will also block all Google’s other user-agents.
Google Bot Bling is WordPress Plugin is designed to help website owners know when Google has indexed their website. This wordpress plugin mails you everytime google crawls any webpage on your site. If you are using adsense then in every 30-40 minutes google crawls your blog.
How To Use Bot Bling?
1. Go to this link and download the plugin.
2. Install it to your wordpress site.
3. After installing and activating the plugin, go to settings > google bot bling and add your email address where you want to receive the notifications.
Note: Google will crawl your site many time, may be you’ll get too many email notification, i will suggest you to create an alternate email where you can check your crawled request or migrate the emails to the spam.