[Gllug] Mature (web)app deployment/management frameworks?

[Walter Stanish]

After many years of manual deployment / maintenance / upgrades of 
(web) applications (ranging from simple to extremely complex with many
dependencies), I now need to start getting in to automated deployments
of fairly complex web systems.  Fairly complex meaning - database 
state, numerous version-sensitive system-level package dependencies,
multiple daemons, external server RPC dependencies, etc. 

If anyone on the list has positive experiences with any solutions in
this area I'd like to hear them.

To give you an idea where I'm coming from:
 - the target environment is two platforms, 90% Gentoo Linux + 
   10% FreeBSD (normal FreeBSD, not Gentoo-FreeBSD)
 - have been doing LAMP stuff >10 years 
 - culturally I favour configurability/flexibility/longer learning
   curve over straight-jacketing tools (just FYI - please no distro
   wars - I migrated to Gentoo from Slackware some years ago, after
   trying many other distros (no names!) and finding their package
   management painful, broken or insufficiently flexible)

My goals for an automated deployment/configuration management system,
in rough order of priority.
 - deploys cleanly to either a new (or existing) virtual machine, 
   Supporting both virtual hosting or single HTTP domain configuration
 - automated dependency resolution (ie: should probably integrate with 
   the system-level package manager).  Some of the web systems I work
   with integrate with subsystems as diverse as email/VOIP/SMS/payment/
   XMPP/image + video transcoding/shared auth databases/etc... too
   much to approach efficiently without automation, so this
   is an absolute requirement.
 - ideally allows rapid addition/remote management of custom
   configuration options, including real-time validation + adequately
   verbose error reporting (save time locating the cause of issues)
 - ideally able to control services after setup (configuration editing,
   daemon start/stop, potentially even debugging/statistics) 
 - ideally the ability to auto-generate a (filesystem for, or complete)
   virtual machine that can be instantiated automatically for testing
   via xen (or similar) ... an achievable pipe dream! :)
 - ideally not tied to one HTTP daemon
 - probably as a separate system: the ability to auto-generate
   deployable VMs or package(-collection)s after particular versions
   of code have passed testing and are deemed to be stable.  Ditto
   for test-environments.
   
If anyone has any recommendations on anything in this area I’d be glad
to hear them.  I'd assume that autobuild systems for larger pieces
of software may include something along similar lines.  Maybe the open
source vhosting (VHCS/ispconfig), configuration management (puppet)
and network monitoring tools (nagios/SNMP) would also offer some subset
of required functionality.

Right now I'm thinking something along the lines of...
 - convert webapps to Gentoo packages, which will handle system level
   dependencies very well and provide a mature install/upgrade script
   environment
 - use the Gentoo webapp-config tool (I haven't been using this in the
   past due to the infrequent + custom + complex nature of my webapp
   installs, plus I'm not sure how complete/maintained it is these days.
   I'm *very* interested in alternatives here, maybe open source vhost
   management tools that don't make too many distribution or OS-level 
   assumptions?)
 - use Xen for virtual machine creation with virtual networking for any
   generated VM (<buzzword>rapid provisioning</buzzword>)
 - use rrdtool / nagios for statistics + monitoring
 - use sshkeys to automate remote deployments/upgrades/queries with
   custom tools which will be developed as required
 - potentially write a separate packaging server to bridge existing
   SVN environment to new, stable/deployment-ready package database
   (probably another SVN server), then bridge this to clients via
   Gentoo's emerge 'overlay' feature.
... however the above will scratch FreeBSD as a platform.

As this is a fairly major infrastructure decision I would like to make
ASAP I'd like to hear feedback/experiences/recommendations/curses from
fellow list members who are using systems that are operating in any
(subset of) related areas.

(Sorry for the slightly vague description.  I suppose network level
systems automation / management is one of those fairly large, time
saving, habit altering 'jumps' that experienced list members will
recognise in their own journey through UNIXland :)

Looking forward to some interesting comments/experiences/recommendations,
Walter
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug