As I was looking through my sitemaps profile this morning, I checked out one of my domains and noticed something very odd.
Looking at the image above, you notice my crawl rates are high, high, high, and then suddenly they just dropped out of no where. With a maximum of almost 6,000 pages a day being crawled to down to not even 600 pages per day being crawled.
Another huge factor I noticed for the same site in the image above is that my search rankings, where I was ranked #2 for the keyword, is now ranked #3. I had been #2 for quite a while and then just suddenly just changed. I checked a few datacenters and it had shown me at #5 for that keyword.
I wonder why there was a sudden drop in rankings for my site. I know my backlinks are strong and haven’t changed — if anything, they’ve increased. Is it possible that too many backlinks is back for your site?
December 7th, 2006 at 8:13 am
How many pages does your site have? Do you use Sitemaps? Could it be that Google is just getting better at keeping up with your changed pages and doesn’t have to randomly crawl that much anymore?
December 7th, 2006 at 11:01 am
I have 8000+ pages in the sitemap. The freq change has been the same throughout and hasn’t changed…so I’m not sure why it would change how fast it crawls.
February 1st, 2007 at 3:29 am
we have about 11302 pages and we getting only 200 - 300 pages crawl by googlebot each time and not crawling all the pages we have stopped updating the website. can you please tell me how long it takes to crawl all the pages?
please check out site map at http://www.bizcento.com/sitemap.php
help us in this regard.
February 1st, 2007 at 9:53 am
I personally don’t know, someone who would be better to look at this would be Matt Cutts, but my guess would be that you just need more backlinks so Google starts indexing more and crawling more.
February 2nd, 2007 at 12:22 am
Thanks Mr.Jonathan,
We have visited some time ago with mattcutts seo blog and find some info and we have asked Mr.Matt as well. we didn’t find anything about it the crawl. Anyhow thanks once again.