[Durham] File sync, large scale

[Lucky Lee]

Evenin' all.

I have a need to stand up two Gluster clusters (likely multiple volumes) in
different physical locations and sync data between them in both directions;
effectively, they'll mirror each other at a file level.

Latency of some minutes between a file appearing one side and being sync'd
the other side is acceptable (by that I mean I don't need synchronous
writes or updates within seconds).
I've got around around 1Gbps of usable bandwidth between sites.

I did consider Gluster geo-replication but that is one-way.

In terms of scale, there's around 20 million files and approx 1Tb of data.
Some of the data (tens of thousands of files) will change at least daily,
but the vast majority will be unchanged/static from day to day.

I'm current considering syncthing & unison.  Any other recommendations?

Thanks in advance,
-Lee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/durham/attachments/20200301/46fdeb42/attachment.html>