[Phpwm] Problems with live Silverstripe website (httpd spawing proceses, high CPU)

[Paul Matthews]

Perhaps it's someone's spider they've written / downloaded, especially if
you say it's following a normal pattern. They could have been spidering the
Indian site and have followed a link to the UK site.

If your site does have issues when it's being crawled, although unlikely
that it'll be anywhere near 1000 request per second, then perhaps something
needs to be done to improve performance?

David, is that apache plugin likely to blacklist google when it tries to
index a site? as that would be a clear downside.

Regards,

Paul.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/phpwm/attachments/20100820/010a6f5b/attachment.htm>