[Watford] help replicating large amounts of data

[M Fernandes]

Hello Rob / Steven,

Whilst I am in no way as experienced as either of you in Linux (you may
have seen one to two of my (comparatively) basic questions recently) but,
I've come across the following.  Rob I have no visibility and/or needs of
your setup of course, but, a-sharing we should be!  There's a review here
http://www.linuxjournal.com/article/7712 and the actual site is here
http://www.cis.upenn.edu/~bcpierce/unison/ .

I hope that helps you?  Anyway, back to my own Gentoo dilemmas.  I never
knew pain could be so good!  ;-)

Mike

On 13 June 2012 08:39, Steven Acreman <sacreman at gmail.com> wrote:

> I don't think there is anything better than rsync. It was written by a
> genius.
>
> Just use the faster options like timestamps over md5.
>
> Sent from my iPhone
>
> On 13 Jun 2012, at 08:33, Rob Jefferis <rob at letchmore.co.uk> wrote:
>
> Hi all.
>
>  Has anyone got any ideas on the best / quickest way to replicate large
> volumes of data between servers.
>
>  We currently have 2 NAS boxes and would like to replicate 1 to the
> other. The underlying technology for the sync is rsync and I dont think
> this is really going to handle the volume of data.
>
>  I have just done a count up on one of the larger shares and it is
> currently around 13TB with 1.5 million folders and around 100 million files.
>
>  We have toyed with building some large Windows file servers and using
> DFSR but I dont think this would handle it either.
>
>  Anyone got any ideas on software / hardware that might help? I would
> estimate the total required storage at primary to be around 50TB so a
> guestimate would put that at around 350 million files.
>
>  The data needs to be available to Windows machines.
>
>  Thanks
>
>  Rob
>
> _______________________________________________
> Watford mailing list
> Watford at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/watford
>
>
> _______________________________________________
> Watford mailing list
> Watford at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/watford
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/watford/attachments/20120613/e1830ee7/attachment.htm>