Control Your Googlebot Crawl Rate

Posted by jonathan at 8:22pm EST on 01/20/2009

Google introduced a new crawl rate control panel into the Google Webmaster Tools.

These settings will give you more control over how quickly and how frequently Googlebot crawls your site and pages.

In most cases, you will want to keep it at the recommended setting, default, but in a rare circumstance — such as you feel your site is not being indexed fast enough or the content is not as fresh as it should be for your results, you may want to alter these settings.

Be aware though, by changing it to the fastest setting, it’s never going to guarantee that Google will index and display your content faster, it’s simply a preference that you are giving to Google. A key factor to helping get your content indexed faster is still backlinks, backlinks, backlinks.