[Gllug] Hacker Attack

[Bruce Richardson]

On Wed, Jan 11, 2006 at 10:20:43PM +0000, Mike wrote:
> On Wed, 2006-01-11 at 18:05 +0000, Bruce Richardson wrote:
> > 
> > People who know how to compile kernels but don't consider the
> > implications of installing them have caused me quite specific pain in
> > the relatively recent past.
> 
> I'd be interested to hear you expand on that. I compile my own kernels
> for production servers, and maintainability is a consideration. I find
> Debian (as ever) has removed a lot of my worries - the infrastructure
> provided by kernel-package is excellent, and nicely keeps the config,
> kernel and bootloader working nicely. That said, I'd be most interested
> to hear what in particular has caused you pain when picking up
> afterwards...

Well, even where a kernel was built using kernel-package, it isn't
necessarily easy to tell precisely how it was built just from
/boot/config and the auto-generated files in
/usr/share/doc/kernel-image.  Any number of gruesome and unstable
patches might have been applied with little or no external evidence.
This would be an annoyance if discovered on any infrastructure host but
a bloody nightmare on crucial HA systems which you had hoped to replace
in a seamless, staggered fashion but couldn't because the unstable,
arcanely-patched kernels would not work with anything but themselves.

Stripped-down kernels built with no documentation *and* no support for
modules (and, of course, no thought of a recovery strategy) are a
special joy when the old hardware they were built on breaks.  Put the
hard drive into something else and it won't boot.  Add a new kernel and
it will boot but you now can't see how the old system was functioning.

Then there is the "spare kernel", where some bright boy installs a new
kernel on a system without doing anything to verify that it is suitable.
Does it provide all the required features?  Will it even boot?  Nobody
knows, but the magic of package management and automatically managed
grub configuration is likely to have made it the default kernel and a
lovely surprise for somebody at some unknown point in the future.

Of course, systems should never be built like this but in fact it is
quite common.  A significant number of people are smart enough to roll a
custom kernel but not so bright as to think beyond that point.  If they
were only responsible for their own workstations, this would not be a
problem.  The IT industry being what it is, many of them are in
well-paid Sysadmin jobs.  While they remain in post the damage may not
be too bad, because they remember these little details (assuming the
systems haven't become too complex and chaotic for them to keep track).
Indeed, they take pride in all these little details, mistakenly assuming
that the more tiny details that they are tracking, the more applications
they custom-build for imaginary performance gains, the more they use
bleeding edge and experimental technologies in untested alchemical
combinations, the more skilled they must be.  Pity the unfortunates who
have to pick up the pieces after they have moved on; something is bound
to break before they have a chance to do a proper audit and fixing it
will then be expensive.

Rolling custom kernels can give very definite benefits if done with
some thought.  Building custom application installs from source is less
often useful (most distributions provide perfectly good packages that
not only do the job but are usefully integrated with the rest of the
system) unless an organisation has put some time and resources into a
decent build system.  A little thought and planning should always be
done beforehand if you are managing anything more than your own
workstation, since every change you add makes the systems a little more
complex (and requires extra documentation.  You are logging and
documenting your changes?  Say yes and I'll put the gun down).
Unfortunately, these things are done all the time by people who have
given it no thought at all but who do it on principle, because they
think that a) these activities have intrinsic value and b) that it shows
how skilled they are.

-- 
Bruce

I see a mouse.  Where?  There, on the stair.  And its clumsy wooden
footwear makes it easy to trap and kill.  -- Harry Hill
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: Digital signature
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20060112/5489db8e/attachment.pgp>
-------------- next part --------------
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug