Talks (was: Re: [YLUG] A basic question)

[Steve Kemp]

On Fri Dec 28, 2007 at 18:54:08 +0000, Alex Howells wrote:

>   *  Scalable spam prevention  -  anyone with a working solution
>       capable of handling *lots* of mails per day?  What software are
>       you using to accomplish it?

  QPSMTPD[0] is what I use as a framework for handling spam.  This is
 a flexible plugin-based SMTP proxy server written in perl.  It allows
 pretty much every aspect of the SMTP conversation to be processed, and
 allows things to be dropped early very easily.

>       Any particular "tweaks" you came
>       across during testing which weren't well documented?

  I've run into two distinct problems which have the same cause,
 validating "normal" fields is hard.  There are people who'll just
 abuse the mail format.

  Specifically I tried to:

   1.  Validate the 'Date' header on mails, if present.

   2.  Attempt to deny "foreign" messages based upon an examination
      of the character set used.  (I'll save the charset rants for
      another time...)

>       Is there an individual quarantine per user/domain, perhaps
>       accessible through a web interface?

  Ahem.

>   *  Scalable web hosting  -  deploying WWW services behind load balancers
>       and utilizing technologies like caching on dynamic content.
>       Would be particularly interested in hearing from someone who's
>       deployed a Ruby on Rails application in such a manner, given that
>       it appears such a c*nt to deploy in any sane fashion :-)

  I've no experience of large-scale hosting, but I do highly recommend
 fragment caching via Danga's memcached or similar software.

  My Debian site uses it extensively and I'd be happy to show you the
 cache hit/miss rates if you're curious.  In short when you have per-user
 pages being generated on the fly caching them doesn't work out as well
 as it could - because too many parts (potentially) change.
  Caching the computationally intensive sections is a significant
 win.  (Probably pretty obvious..?)

>   *  Firewalls - minimo or maximo?  Anyone want to play devil's advocate
>       on whether all systems should run a firewall? :)  Prepare for heckling.

  Firewalls are a panacea for 90% of users who neither know what they
 protect against or how to use them correctly.  ("Oooh I have a firewall
 I'm protectored!")

  A)  No Services == No need for incoming firewall.

  B)  No outgoing firewall == No firewall.

  C)  When attackers can run arbitrary code upon your machine it is game
     over anyway..

>       For those systems *with* a firewall, do you `-P INPUT DROP` and go
>       from there; or is your preference to operate just a blacklist?

  I've done that recently, for fun, and ditto for outgoing.  It is
 pretty easy to get it right if you keep an eye upon your machine, and
 know what should be running/reachable.  Definitely log all DROPs before
 declaring it complete - or things will break when you attempt to do
 things like rsync upon ports like 9898.

  Also note that iptables, the firewall backend, doesn't do UDP
 terribly well..  (e.g. forwarding/dnatting of X11 login sessions.)

>       If you don't advocate a firewall for the average system, why?

  See above.

>  Any tips or tricks to speed up or automate the process; any tips or
> tricks to "managing" the whole shebang so key trains of the process 
> aren't dependent on one guy/girl?

  The only thing that I'd say here, with a nod to Joel On Software, is
 that your "product" (be it a commercial program, another piece of
 software, or a release of a complete distribution) should be released
 via a single tool.

  If you can type 'make release', '/home/$foo/release-me.sh
  --version=2.0.1' then you've got it made.

  If you cannot then you're at the mercy of the people who can do
 the magic.  I've watched large projects switch to this model of
 allowing easy release of:

    * Final builds
    * PDF documentation.
    * Release notes.
    * Source Control tagging / etc.

 It really is key.  In a prior life I watched an increasingly frantic
 last-minute release of a version of our software for a trade show.
 Too many manual and magic steps meant that the application of a
 critical last-minute bugfix which should have involved <5 lines of
 code change and a rebuild took almost 48 hours to release.

  (I ended up carrying the CD-ROM upon a plane with me ..)

Steve
-- 
# Commercial Debian GNU/Linux Support
http://www.linux-administration.org/

[0] - http://smtpd.develooper.com/