[Nottingham] Parallel rsync?

[James Moore]

No idea here either, but a thought to advantage: such a setup *might* 
work for synching many tiny files (less than a couple or three sectors 
each, maybe?), it'd kill the system if you're dealing with large 
binaries. From experience in populating flat-file databases on commodity 
gear, I would say keep to as few threads as possible to minimise I/O 
priority switching hence head wear. I've killed a RAID in testing 
running multiple copy threads into the array using test data of 
millions(!) of random and small files (1B<5KB) for the small problem of 
excessive heat and vibration.

On 29/07/2013 13:01, Dan Caseley wrote:
>
> I've no idea how to achieve this, but i thought I'd ask for my own 
> curiosity - what's the advantage to running parallel rather than 
> series? Isn't the pipe going to be the limiting factor here in any 
> scenario?
>
> Dan
>
> On 29 Jul 2013 11:14, "Jason Irwin" <jasonirwin73 at gmail.com 
> <mailto:jasonirwin73 at gmail.com>> wrote:
>
>     Other than a script to loop over folders and start multiple
>     instances of
>     rsync...does anyone know of an easy way to get it to do a
>     multi-threaded
>     copy? i.e. copy in parallel.
>
>     FYI: I'm having to use Cygwin on Windows on the server side, and I'll
>     probably have to use the same on the client.
>
>     --
>     ??????????????????????????????????????????????????????????
>     ? Jason Irwin ? OpenPGP (GPG/PGP) Public Key: 0xD0C592B1 ?
>     ?             ? Import from hkp://subkeys.pgp.net
>     <http://subkeys.pgp.net>        ?
>     ??????????????????????????????????????????????????????????
>
>     _______________________________________________
>     Nottingham mailing list
>     Nottingham at mailman.lug.org.uk <mailto:Nottingham at mailman.lug.org.uk>
>     https://mailman.lug.org.uk/mailman/listinfo/nottingham
>
>
>
> _______________________________________________
> Nottingham mailing list
> Nottingham at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/nottingham

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/nottingham/attachments/20130803/4ca27768/attachment.html>