[Gllug] squid, load balancing

SteveC steve at asklater.com
Mon Jan 16 13:32:28 UTC 2006


* @ 16/01/06 01:52:21 PM rich at annexia.org wrote:
> On Mon, Jan 16, 2006 at 10:54:30AM +0000, SteveC wrote:
> > I'm looking for advice on scaling a website (openstreetmap.org) above
> > one box as we're hitting quite high load. The current topology is a db
> > machine, the web server www, a tile server for imagery and a machine
> > with backup harddrive space. Everything runs debian and apache2.
> 
> I suspect that first of all you need to do a bit more analysis to find
> out where the bottleneck is.  I had a quick look and it looks like
> your code is written in Ruby, using mod_ruby and sending the right
> expires and cache-control headers (presumably those come from an
> instance of Squid in front of each web server).  This is all good
> stuff.

There are multiple bottlenecks. The first is that we're running on
essentially donated hardware so it's all a bit crappy, not enough memory
etc. We have had a surge in users:

http://www.openstreetmap.org/wiki/index.php/Stats

putting stress on things. Our application (free mapping) means throwing
a lot of data around and we don't have the space to pre-generate tiles
hence squid sitting in front of apache on the tile server. 

> Could the problem be the database server?  If all your tiles are
> stored in the database (I don't know if this is true or not), then
> having the separate tile server may not be helping you at all.  As
> suggested by others, try instrumenting the database (if that is
> possible) to see which queries take the longest to run, and then
> optimize those by adding indexes.

It's MySQL currently. Every query we do has an index and it's pretty
fast.

have fun,

SteveC steve at asklater.com http://www.asklater.com/steve/
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug




More information about the GLLUG mailing list