[HLUG] Online Linux Backup Utilities - A Review

[Julian Robbins]

Hi

I've been investigating some apps for online data backups recently. The
requirement was for the following:-


   - Open Source
   - Robust and easy to use
   - Offers Encrypted Backups
   - Can handle Backblaze B2 storage
   - Uses Block storage ie doesn't need the older Full, Inc, Inc, inc, Full
   backup regime any more
   - Not too many library requirements as working on a headless Ubuntu
   16.04 server
   - Cmd line or GUI is fine.


When you look into this there are actually tens (perhaps hundreds) of
passable Linux backup Applications/Scripts
<https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs>.
But many are either unmaintained or have a particular unusual case.

After reviewing a number of applications I narrowed to the following:-

   -
   - BorgBackup <http://borgbackup.readthedocs.io/en/stable/>
   - Duplicati <https://www.duplicati.com/>
   - Duplicacy <https://duplicacy.com/>
   - Duplicity <http://duplicity.nongnu.org/>
   - Restic <https://restic.net/>

*Borg <http://borgbackup.readthedocs.io/en/stable/>* is very well reviewed
and used widely, and offers encrypted backups and is a mature piece of
code. It's cmd line only but this doesn't matter me, less to go wrong !
Unfortunately it only works with local or SSH backends so I cannot use it
directly with B2. That said there is another useful tool called RClone that
is a library that works with your backup utility and has many backends
(approx 30) that it works with, so you can use RClone to connect B2 or
others such as Amazon S3 with Borg. If you wish to do Offline and Online
backups this may be ideal, but this wasn't part of my requirement and I
felt it added more complexity and more to go wrong. I like my backup
application to be simple but robust in design and operation. So Borg was
out for me (at least for now). Certainly a great backup application.

*Duplicati <https://www.duplicati.com/>* is another favourite of mine. Vers
2 beta which is a complete rewrite of the backing up algorithm to a block
based system, has been in the works for about 5 years so, for a Beta
(officially) is actually *very* stable. It's one I've tried for other
projects and its web gui is particularly useful. But a downside is the
requirement for up to date Mono packages which is a problem if running with
Ubuntu 16.04 server which is nearly two years old now. I struggled to get
this to run on this server, and it's probably more suited as a Desktop
application. I've found Duplicati a little tiny bit buggy in the past, but
the actual backups have been very reliable; just other parts of the
application have been a little flakey at times. In this case the
installation proved to be too unwieldy for a server application.

*Duplicacy <https://duplicacy.com/>* is related to Duplicati and Duplicity.
Yes it is very confusing ! The reviews are very promising and it does
encrypted backups well and its backups are made quicker than the other
applications listed. One big downside is the confusing licencing where the
source code is open, but compiled binary is not and for commercial use you
need to pay a small fee. I prefer my applications true FLOSS, so this was a
big mark down for me. There is also a more limited Gui version of this tool
too.

*Duplicity <http://duplicity.nongnu.org/>.* This is one I had used about
ten years ago and is a great application that is rock solid and very
mature. It features encryption and can write to many backends both local
and remote ones. The main downside is that it operates not with a block
based storage algorithm but requires a Full backup, then a number of
Incremental backups based on the Full and each Incremental. Then another
Full after a relatively shortish period of time. The reason you need to
keep regularly taking Full backups is that if one part of the original Full
or any Incrementals is corrupted it breaks the backups meaning that the
chain is broken. Therefore there is a necessity of taking Full backups
fairly often. In the days of tape backups this wasn't a problem per se, but
with ever growing backup datasets and charges of using online storage and
traffic, and the sheer amount of time to retake a Full, I had to take this
this application off my list. A Full backup of approx 600GB of data was
taking over 6 days to backup over a fibre FTTP connection. And this would
need to be redone every time a Full was required. If these complete Full
backups are made there is also a greater chance of a power glitch or
network hiccup over such a long time period of several days.

*Restic <https://restic.net/>*

Saving the best (for me) till last. Restic is very much along the same
lines as Borg but with a smaller feature set and slightly shorter
development time for the project so far. It features all my requirements
such as encrypted backups, block based so only the initial Full is required
then Incrementals after that (called Snapshots in their wording). The
application is well designed, its encryption appears from peer review to be
particularly well implemented, and the application developers have learnt
wisely from other projects in their design choices. The major advantage
that Restic has over Borg is that it can write to a number of different
online backups such as S3 compatible or B2 online storage, etc out of the
box. It's written in Go and requires hardly any other libraries to be
installed. In fact i had it installed and had taken my first encrypted
backups in only a few minutes. The developers are very welcoming and work
with and listen to their community which is a nice change from some FLOSS
projects. Its only downside is that it's still a newcomer in many way
celebrating its 4th birthday recently compared to many of the other
applications. But what it does, it does really well in a clear and concise
format.
Incidentally, Restic have recently announced they will be using RClone to
increase the number of backends it can write to which I think is a great
example of similar projects working together for the greater good.

I hope this will be of interest if you fancy trying to backup your data
remotely in the cloud. Its far easier now than it ever was and there are
numerous great solutions available for Linux.

When I get a moment I will upload this to our website ..

Thanks
Julian