Google backs off

15th November 2005

Since I setup Google Sitemap and robots.txt across my sites the GoogleBot has become slightly less intense, and is now only responsible for about 25% of the bandwidth that the sites all use! I still think that the spidering is a bit intense for personal web sites, and it seems not to be taking much notice of the sitemaps recommendations, but it is now down into the realms of acceptable behaviour.

tags: blog home computing web

Post new comment

The content of this field is kept private and will not be shown publicly.

Archive

User login