[Gllug] Production system - Linux 2.4.24, LVM and cciss

[Simon A. Boggis]

On Mon, 2004-01-12 at 11:10, Dale Gallagher wrote:
> Hi Simon
> 
> You have made some very pertinent points!  I definitely need advice at
> this stage, as the solution I'm building is quite ambitious and
> reliability/backups are vital.  As soon as I have the budget, I'll
> invest in an appropriate tape solution.

Abso
> ..
> 
> > You're considering RAID with high redundancy for your filesystem so I
> > presumably don't need to tell you how likely and often disks fail -
> > your proposed system is twice as likely to fail as a single disk, and
> 
> but more resilient overall really... if a disk (or two under
> particular circumstances) fails, the array continues without data loss,
> whereas the loss of a lone disk leads to data loss _and_ downtime.
> 
> > if it does you loose 50% of your backups instantly.
> 
> How so?

Sorry, that was really badly worded (early morning email <;) - I was
meaning that assuming you have two hotplug SCSI disks to do your backups
to, if one fails you must loose 50% of your backups assuming you
distribute backups evenly.

Also, unless you are going to exchange them every day, I assume that
you'll do the obvious and backup for several days onto one disk; if so
then when you plug it back in and it fails to spin up you'll not only
loose 50% of your backups but unfortunately all the most recent ones
might well be on that disk.

If you split your backups across two disks the chance of failure is
double the chance of a single disk failure; I failed to note that the
result of failure is half as bad (you loose 50% of your backups as
opposed to 100%) as a single disk, which is remiss of me.

I also failed to take account of you doing many archives onto one disk,
which was me being daft (see earlier mail to Huw).

My feeling however, is that if you carry this to its logical conclusion
by the time you've got several 10s of disks and are now loosing little
on a single failure you'll unfortunately have also garnered a quite
considerable risk of failure because the failure probability on disks is
quite large, whereas you can own 100s of tapes and do the same because
they're inherently more robust and reliable and have a lower failure
probability.

Hmmm, am I arguing myself in circles here?

> For now, I'll run incremental daily data backups and bring one of the
> disks off-site once a week and backup this backup to IDE-disk at the
> office. As the user-base increases to a point where this is no longer
> possible, realistically viable, or I have the budget, I'll upgrade to a
> tape backup solution.  I'll be backing up system files to CD once a
> week (the box will have a DVD/CD-RW drive), or after significant changes
> to the system, whichever occurs first.

Growing into something sounds absolutely reasonable - I wouldn't dream
of saying an absolute "Don't do it!", but hopefully (helpfully?) point
out some reasons, costs, drawbacks and benefits of other approaches.

For example, despite my dual backups to disk and DLT or LTO at work, at
home I do no such thing because I can't afford it: I rely on mirrored
IDE disks, rsync-ing important stuff to various machines and reasonably
regular backups of important data to CD writables (and soon a cheap DVD
writer I hope), and I figure that if I get unlucky I'll "pay" the cost
in my time in reinstalling my OS.

As a second example, we (mostly) don't do archival backups (meaning we
cycle all our tapes, without keeping some by for long term archives). We
do occasionally make archive sets now and then as required. My superiors
didn't want to pay the costs of doing regular archival dumps, because it
would mean owning many more tapes than we do, but consider a reasonably
long cycle of tapes sufficient for their needs. I advised them on the
risks which come with this approach, but it is their data, money and
their call at the end of the day.

> 4 disks in RAID1+0 config (slots 0-3)
> 1 hot spare in slot 4
> 1 Hotplug AIT drive in slot 5 (available in the HP-DL380)
> 
> With daily and weekly tape backups.  Any comments on this and my current
> (interim) backup plan, now that you have more detail?

I'd feel happy with a daily backup cycle to tape, and if you can afford
a longer, slower cycle that's lovely too - I don't have experience with
AIT itself, as I chose LTO quite early on for various reasons, but
provided you check it is all happy with linux (I'd guess that it is -
the only thing I was aware of a while ago was that you didn't get
support for that on-tape-cartridge-volatile-memory-index-storage).

Regards,

Simon

-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug