Seems these past days (or is it just wishful thinking since Feb. 18th) that both crawlers are pretty active.

Through an IP identified as inktomiseach.com, the U.S. Slurp has been frenziedly slurping my pages at a rate of approximately once every hour. MediaTic reported yesterday that his site was crawled, with as much interest, by the French Slurp.

Unfortunately, the URL he gives doesn't work anymore. I tried to check if Yahoo!France gave any information as to it's robot. No explanation given. None either at the Yahoo.com search help file, even if it states that "there is a Yahoo! Slurp-specific extension to robots.txt which allows you to set a lower limit on our crawler request rate."

Googlebot
It used to be a pain to get that crawler to regularly visit one's pages. No more! I get visits from Googlebot twice a day. My meta tags have been set, from the beginning, to encourage robots to revisit my blog every 24 hours.

Moreover, the MSN bot has also started to visit more than once a day.

Don't get me wrong here, I'm not complaining. But the only thing I don't understand is why the search results, for both Yahoo, Google and MSN, aren't updated more frequently.

As of this morning, Yahoo and MSN show my latest post being from February 20th and Google dates back to February 19th. If anyone has a logical explanation…

Categorie(s)/Catégorie(s): Blog musings , Tech and Web

Feedback/Réaction(s): TrackBack & pings(0) | Comments/Commentaires(0) | Referring Web Pages, last 5 days