[YLUG] more hosting chatter

[Matthew Gates]

All this talk of hosting has reminded me of something which I would like to 
find out about.  I've had a very cheap hosting solution for a few years, and 
it's all well and good for low traffic personal stuff.

However, I need to find out about hosting with different requirements - 
essentially what would be necessary and in what price range it would be.

The task is to serve something like tiled map data (actually image tiles 
from an astronomical survey - think the "sky" part of google earth).  The 
data set size is something of the order of a few gig.  It's all static 
files.

Typical user session:
- connection, fetch 1 large-ish file (300 kb)
- two to four download threads, probably working 85% of the time during a 
user session, dragging files from the server as fast as the connection 
allows.
- typical download thread behaviour:  
  + download a group of 2-16 very small files in quick succession (a few 
hundred bytes each), 
  + then grab a similar number of image tiles (2-50 kb).
  + repeat until end of session.

The number of simultaneous sessions is hard to judge.  I estimate between 30 
and 250 concurrent sessions at any one time, with occasional spikes of up to 
2000 concurrent sessions (but rare enough that it doesn't matter if it 
really hits performance).

One requirement is that this is done with fairly low latency.  I suppose any 
hostco tries to optimise their systems to do this, but I've found the 
cheaper virtual host solutions can be pretty slow sometimes - especially the 
connection time.

It seems to my inexperienced mind that this is a lot of open connections at 
once.  Anyone out there have some idea of what this sort of solution would 
cost, or the best way to go about it?  I'm wondering if it's possible on a 
cheaper virtual host, or if it would require something more expensive.

The file structure is atypical for a web site - there ar many directories 
with thousands of tiny files in.  This makes be think that filesystem choice 
could affect performance.  e.g. reiserfs is _much_ faster than ext2/3 for 
handling large numbers of small files.  I guess that would require a lot 
more control of the server than a typical virtual hosting solution...

Cheers
Matthew

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.lug.org.uk/pipermail/york/attachments/20081112/3abe7388/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
Url : http://mailman.lug.org.uk/pipermail/york/attachments/20081112/3abe7388/attachment.pgp