[Gllug] squid, load balancing

[Richard Jones]

On Mon, Jan 16, 2006 at 10:54:30AM +0000, SteveC wrote:
> I'm looking for advice on scaling a website (openstreetmap.org) above
> one box as we're hitting quite high load. The current topology is a db
> machine, the web server www, a tile server for imagery and a machine
> with backup harddrive space. Everything runs debian and apache2.

I suspect that first of all you need to do a bit more analysis to find
out where the bottleneck is.  I had a quick look and it looks like
your code is written in Ruby, using mod_ruby and sending the right
expires and cache-control headers (presumably those come from an
instance of Squid in front of each web server).  This is all good
stuff.

Could the problem be the database server?  If all your tiles are
stored in the database (I don't know if this is true or not), then
having the separate tile server may not be helping you at all.  As
suggested by others, try instrumenting the database (if that is
possible) to see which queries take the longest to run, and then
optimize those by adding indexes.

> We're using squid to do some caching but it really isn't fun to
> configure from my limited experience. Are there other things to consider
> apart from squid? What are the options for load balancing?

Your current squid config looks fine.  I've also used Apache +
mod_proxy in the past.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug