Since the rise of the container community over the past three years, many companies and projects have emerged that offer truly innovative ways to manage applications.
There are several projects around management, networking, storage, logging, monitoring and more ( see this ecosystem mind map ). Container operating systems play a particularly important role in this ecosystem.
In all the companies I have consulted for so far, there was one question that all developers asked themselves: What is the best operating system to run containers? A complete Linux? CoreOS? What about RedHat? I’ve also heard of something called RancherOS? And all these „Atomic“ systems?
Of course, the answer is as always: „It depends“. Below I list container OS and their features:
CoreOS
CoreOS focuses on large-scale installations, primarily aimed at enterprises. It has the following advantages:
Hundreds of contributors and usually 500+ IRC users in #coreos on FreeNode. It comes with some really interesting tools developed by the CoreOS team, like etcd, fleet and flannel. Using these tools you can set up a CoreOS cluster very quickly.
In December 2014, the CoreOS team also announced another type of container runtime engine called rkt. This was a response to what the team saw as a drift away from Docker’s original container manifest. CoreOS still runs both Docker and rkt containers, so there is no need to worry about compatibility yet or in the near future.
The CoreOS team has also partnered with Google (Google Ventures is also an investor in CoreOS) and has developed Tectonic, a really interesting way to get CoreOS running with Kubernetes easily and efficiently. Tectonic is a commercial Kubernetes platform that can be important. And of course the acquisition of CoreOS by RedHat also has an impact, because RedHat 4.0, for example, runs on the Red Hat Enterprise Linux (RHEL) CoreOS operating system. A reduced, immutable OS optimized for running containers. This will probably replace RedHat’s Project Atomic sooner or later. Project Atomic or RHEL Atomic Host is also an immutable and reducible OS based on RHEL 7.
RancherOS
Rancher tries to consistently follow what a container operating system should be. Everything in RancherOS is a Docker container; it runs a system Docker as PID 1 and then starts a container for each user.
This may seem crazy, but for an operating system that wants to do nothing but host containers, it makes sense. Rancher has removed everything that is not needed. This makes the OS very lightweight. The installation ISO is only 22MB.
Even more interesting are all the services that are added with the Rancher system beyond the operating system. If you consider what is needed in a container production system: Typically functions such as security and networking, service discovery, load balancing, monitoring and scheduling. Rancher adds all of this on top of RancherOS.
Snappy Ubuntu Core
This is an interesting project that Mark Shuttleworth announced last year, while also calling some of the container operating systems available at the time too „bloated“.
The Snappy Ubuntu Core OS comes with a new kind of application manager („snappy“) and focuses on running applications and containers. Some may argue that this is contrary to what a container operating system should do, but it could also be a good transitional system. It represents a great learning opportunity for those who just want to try things out and don’t have time to learn the intricacies of etcd, consul, fleet, kubernetes and all the other tools.
The basis of the system is the „Ubuntu Core“. In addition, apps exist in read-only images (similar to containers), and the apps can be updated „transactionally“. This is great because you don’t have to re-download an entire application to deploy a new version, just the changes made (like with Image Layer).
The Snappy Ubuntu Core OS is not a pure container operating system, but it does have some interesting aspects. Anyone who has used Ubuntu in production or is interested in running both applications and containers side by side should definitely check it out.
RedHat Project Atomic
This distribution is built with upstream RPMs from CentOS, Fedora and RHEL and enables what RedHat calls „atomic“ upgrades and rollback.
The operating system has built-in functionality for Docker, flannel (from the CoreOS team), Kubernetes, a transactional operating system update tool called rpm-ostree that always keeps an older version of the operating system (similar to CoreOS) available, and of course systemd.
Project Atomic uses SELinux for container security and to manage access to and from them. With the acquisition of CoreOS, Project Atomic will likely be absorbed into RedHat CoreOS.
Mesosphere DCOS
Mesosphere DCOS is a project that is unfortunately often confused with Apache Mesos.
It takes the open source projects Apache Mesos, Marathon, Zookeeper and some other services, bundles them together in a clever way and also adds enterprise features. DCOS is offered in two variants: a community edition for AWS workloads and an enterprise version.
The most interesting thing about Mesosphere DCOS is that it is not limited to container management. After all, it is built on Apache Mesos, which can do a lot more. How about deploying a Hadoop cluster? Maybe a large Cassandra cluster? Mesosphere has this functionality built in, and I believe this is one of the key differentiators from the other container operating systems. It feels like DCOS has lost steam compared to Kubernetes and OpenShift.
VMware Photon
VMware Photon is a container operating system and the first part of an ongoing open source series from VMware. VMware is definitely focused on large-scale application deployment, as shown by their other project Lightwave, which provides identity services including authentication and authorization for large distributed infrastructures, applications and containers.
Summary
As you can see, there are now a number of container OS manufacturers and more will surely follow. In the enterprise environment, I would go with CoreOS – especially with regard to OpenShift.
