[Durham] File sync, large scale

[Lucky Lee]

That's a backup emergency option - thank you :)

On Wed, Mar 4, 2020 at 5:13 AM Edmund Fisher via Durham <
durham at mailman.lug.org.uk> wrote:

> Hey,
>
> How about keeping it simple and running rsync as a daemon?
>
> --
> Ed
>
> On Sun, Mar 1, 2020 at 9:28 PM Lucky Lee via Durham
> <durham at mailman.lug.org.uk> wrote:
> >
> > Evenin' all.
> >
> > I have a need to stand up two Gluster clusters (likely multiple volumes)
> in different physical locations and sync data between them in both
> directions; effectively, they'll mirror each other at a file level.
> >
> > Latency of some minutes between a file appearing one side and being
> sync'd the other side is acceptable (by that I mean I don't need
> synchronous writes or updates within seconds).
> > I've got around around 1Gbps of usable bandwidth between sites.
> >
> > I did consider Gluster geo-replication but that is one-way.
> >
> > In terms of scale, there's around 20 million files and approx 1Tb of
> data.  Some of the data (tens of thousands of files) will change at least
> daily, but the vast majority will be unchanged/static from day to day.
> >
> > I'm current considering syncthing & unison.  Any other recommendations?
> >
> > Thanks in advance,
> > -Lee
> >
> >
> >
> >
> > --
> > Durham mailing list   -   Durham at mailman.lug.org.uk
> > https://mailman.lug.org.uk/mailman/listinfo/durham
> > http://www.nelug.org.uk/
>
> --
> Durham mailing list   -   Durham at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/durham
> http://www.nelug.org.uk/



-- 
Lee Blackwell
+44(0)7500 900547
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/durham/attachments/20200304/3e12e4e5/attachment.html>