[Nottingham] A website attack from google?!

Mon May 3 19:29:01 UTC 2010

A quick grep through this weeks logs, I get

crawl-66-249-65-199.googlebot.com - - [02/May/2010:04:45:12 +0100]
crawl-66-249-71-52.googlebot.com - - [02/May/2010:04:41:38 +0100]

on the same website. One is loading an image the other is loading an 
html page. It is possible they use different machines for image and html 
maybe, I dunno. Still, I think we're debating a fairly insignificant 
point of law here, with google hitting me for 250K pages on some days, 
maybe Im just 'special' {:-)

Michael Simms, CEO - Tux Games LTD
http://www.tuxgames.com

On 05/03/2010 04:52 PM, Frederic Vagner wrote:
> Hello,
>
> I agree with you, but according to what I know, Google never (or I've
> never seen him doing it) crawls one website with several IPs at the same
> time.
>
> Fred
>
>
> On Mon, 2010-05-03 at 14:54 +0100, Michael Simms wrote:
>> That isn't correct on the first two counts.
>>
>> Google uses multiple bots on many occasions, Ive seen it for years. Ive
>> had many issues with google (and Ive confirmed it IS google) effectively
>> DOS-ing my website. My site has around 200 pages on it, and there have
>> been times when google requests up to 250,000 pages a day by its bots.
>> While doing it Ive seen multiple google addresses.
>> Also, Google will look at anything it can find. If you put a google ad
>> on a page, google will spider the page. If you use the google browser
>> toolbar thing, google will know about the pages that arent linked to
>> anywhere. Google may or may not make such pages available to search, but
>> you can be a million percent sure it DOES keep the information.
>>
>> Your final point however is perfectly valid, and completely correct,
>> this time it ISNT google. I just felt the need to comment that Google
>> isn't always the innocent party {:-)
>>
>> Well, final point is correct except s/hackers/crackers {:-)
>>
>> Michael Simms, CEO - Tux Games LTD
>> http://www.tuxgames.com
>>
>> On 05/03/2010 01:22 PM, Frederic Vagner wrote:
>>> Hi,
>>>
>>> You think it's Google but in fact,  it's not.
>>> First, Google only uses one IP address when crawling a website.
>>> Second, Google does not crawl pages that do not have links to them.
>>>
>>> So, this is someone trying to hack your website, and if you look a bit
>>> deeper using the IP address, you will find :
>>> 95.211.132.69 resolves to : hosted-by.leaseweb.com
>>>
>>> Google does not use such hosting company ... but hackers do ;-)
>>>
>>> Good luck
>>>
>>> Fred
>>>
>>>
>>> On Mon, 2010-05-03 at 13:02 +0100, Martin wrote:
>>>> Folks,
>>>>
>>>> Hope you've got all your Drupals and Joomlas and whatever else locked down:
>>>>
>>>>
>>>> 95.211.132.71 - - [01/May/2010:19:09:44 +0100] "GET
>>>> /administrator/index.php HTTP/1.1" 301 267 "-" "Mozilla/5.0 (compatible;
>>>> Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla/administrator/index.php HTTP/1.1" 301 274 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /site/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /cms/administrator/index.php HTTP/1.1" 301 271 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.74 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /content/administrator/index.php HTTP/1.1" 301 275 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.71 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /home/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /main/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /portal/administrator/index.php HTTP/1.1" 301 274 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.74 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /web/administrator/index.php HTTP/1.1" 301 271 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /v1/administrator/index.php HTTP/1.1" 301 270 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.74 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /v2/administrator/index.php HTTP/1.1" 301 270 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /j/administrator/index.php HTTP/1.1" 301 269 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /en/administrator/index.php HTTP/1.1" 301 270 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joom/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /Joomla/administrator/index.php HTTP/1.1" 301 274 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla1.5/administrator/index.php HTTP/1.1" 301 277 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla15/administrator/index.php HTTP/1.1" 301 276 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.74 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla2/administrator/index.php HTTP/1.1" 301 275 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla1/administrator/index.php HTTP/1.1" 301 275 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /Site/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /site_old/administrator/index.php HTTP/1.1" 301 276 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.72 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /Site_old/administrator/index.php HTTP/1.1" 301 276 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /cms_old/administrator/index.php HTTP/1.1" 301 275 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.74 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /joomla_old/administrator/index.php HTTP/1.1" 301 278 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /CMS/administrator/index.php HTTP/1.1" 301 271 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.69 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /test/administrator/index.php HTTP/1.1" 301 272 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>> 95.211.132.73 - - [01/May/2010:19:10:01 +0100] "GET
>>>> /backup/administrator/index.php HTTP/1.1" 301 274 "-" "Mozilla/5.0
>>>> (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
>>>>
>>>>
>>>>
>>>> So... Why would Google go sniffing around the admin pages of whatever wikis?
>>>>
>>>>
>>>> It's a jungle out there!
>>>>
>>>> Cheers,
>>>> Martin
>>>>
>>>
>>>
>>> _______________________________________________
>>> Nottingham mailing list
>>> Nottingham at mailman.lug.org.uk
>>> https://mailman.lug.org.uk/mailman/listinfo/nottingham
>>>
>>
>> _______________________________________________
>> Nottingham mailing list
>> Nottingham at mailman.lug.org.uk
>> https://mailman.lug.org.uk/mailman/listinfo/nottingham
>
>
> _______________________________________________
> Nottingham mailing list
> Nottingham at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/nottingham
>