[Swlug] Docker, containers, oh my!

[Matt Willsher]

> On 21 Apr 2017, at 11:53, David Goodwin via Swlug <swlug at mailman.lug.org.uk> wrote:
> 
> 
>> 
>> David - when you say in hindsight you chose the wrong technology with
>> LXC, what do you mean?
> 
> ( I chose over 2 years ago ).
> 
> Since then all the new products/development seems focussed around Docker; LXC looks like it's really only supported by Ubuntu, so I'm skeptical that it has a long term future.
> 
> I can't (for instance) run an LXC container within Kubernetes or AWS's EC2 Container service.
> 
> There aren't anywhere near the same number of pre-built images for LXC like there are for docker.
> 
> LXC doesn't have an equivalent to 'Dockerfile' - so it's not as easy to integrate it into a project.

I think that covers my experience of LXC too - it’s rather like tooling up for building full VMs. While it can do more, it’s a bunch of parts that you need to tool up yourself, creating potential problems with future maintenance and incuring quite a bit of additional cost. Though in operation each container feels like a full OS. 
Ubuntu did product LXD which layers a more cloud-like interface on top of LXC. While they say this should run on other distros, and provide a nova plugin for Openstack to run LXD instances in the same way as VMs. There has been little take up of this tech though (which feels like a lot of the Canonical/Ubuntu initiatives).

> I'm sure people will continue using virtual machines.

I tend to agree here. This is what has caused some of my recent career dilemma. Software is eating the word. There is very much a move to replace general purpose tools with custom code that hits APIs. I think the era of systems administration without coding - and by coding I mean Python, Go, Java etc rather than config management and scripts - is coming to an end.  By extension Linux will look notice like UN*X in a reasonably short time scale and apps be written to be more self contained (like Go, Dropwizard, Springboot in Java land - maybe not all microservices, but along those lines).


> The big 'win' from docker is the images - being able to quickly/easily deploy something in production and know it's identical to what you tested against.

I think this is also the biggest risk - that the images are baked and system is needed to patch them long after then become steady state in production. That patch burden is lessened by the small footprint of running and installed software, but there will still be critical bugs in libraries that will need a full scale redeploy to resolve. Different ways of working. 

> 
> Obviously you could use vagrant/kvm/whatever in the same way.

The images are much bigger though. I think the uni kernel idea may eventually be the way, maybe with clear containers from Intel as a method of providing segregation at the hardware level. 

> 
>> 
>> ...but issues around patching of the base containers and general workflow seemed
>> rather nebulous.
> 
> I used LXC as an attempt to isolate websites from each other. I can't trust the code random designers will upload / host. It was not economic to have a dedicated VM per site ... so LXC was a compromise.

I’ve done a smaller scale system for just PHP apps, and used separate php-fpm processes for each web site, effectively segregation them, though not as fully as having a completely new environment per site. Security then comes down to permissions and with less overhead. 

> I work around update deployment by having a 'master' container image on each host which I update through ansible every so often. Each LXC container is rebuilt from that btrfs snapshot on a weekly basis.

How did you store the data? Separate volume? 

> The problems were :
> 
> * Doesn't scale that well - while btrfs snapshots are very quick to create/run, there's no shared page cache between them, so you need quite a bit of memory ... something like OverlayFS should solves this - but I don't know how stable it is.

I think overlayfs is considered stable enough these days. The alternatives - aufs, LVM, seem to have problems too though. ZFS on Linux is pretty robust, but I don’t know how well Docker plays with it, even though it’s supported.