[Nottingham] NTL Cache Performance Suggestions

[Robert Davies]

On Saturday 01 March 2003 16:16, you wrote:
> On Saturday 01 March 2003 14:13, you wrote:
> > > I like that:  ``Instead of downloading the odd 5MB
> > > file from the mirror, I'm
> > > going to continuously rsync 80GB but never use 99.9%
> > > of it.''
> >
> > ... although, with sarge practically stalled at the
> > moment, the downloads won't even reach that. rsync
> > AFAIK takes large quantities of _server_ CPU time -
> > which is why jigdo was introduced to replace the
> > pseudo image kit
> > (http://www.tldp.org/HOWTO/mini/Debian-Jigdo/whyjigdo.html#WHYNOTUSETHEWH
> >OL EPIK). So, while the bandwidth donor has to pay by the GB,
> > rsync'ing limits the number of concurrent users far
> > more.
>
> Look I just spotted that an FTP server mirror that I have mirrored has an
> rysnc mirror.  Guess what, the 5 minute FTP job, even with little data
> transfer, finished in < 10 seconds.   So in this case 30 times the users
> would need to be allowed concurrently on the ftp server, to match an rsync
> server.  Yes it's an extreme example, but approaches the 'normal case' for
> mirroring applications.

Some numbers to back the point, just tried it again, note there is > 1GB of 
mirror in the 8.1 directory.

time rsync -va rsync://ftp.gwdg.de/pub/linux/suse/suse_update/8.1 
pub/suse/i386/update
receiving file list ... done
wrote 90 bytes  read 78239 bytes  52219.33 bytes/sec
total size is 1165319223  speedup is 14877.24
0.030u 0.160s 0:01.37 13.8%     0+0k 0+0io 210pf+0w

Now the job takes 1.37 seconds, so hard to imagine it used more CPU time on 
the server than my old 10 minute ftp job isn't it.   Perhaps gwdg.de has some 
fancy Quantum effect servers that compute the answers before you ask them to?

Rob