Sashkin - Fotolia
Modern Infrastructure Editor-in-Chief
Published: 17 Sep 2015
What's that rumbling noise you hear? That's the sound of millions of Docker application containers being generated by developers, barreling right this way, straight into the enterprise data center.
Indeed, developers have seized upon Docker with a fervor that hasn't been seen in a while. The application container project and company only launched in 2013, but it is already valued at over $1 billion. Docker Container has been downloaded more than 400 million times, and there are over 100,000 "Dockerized" applications in Docker Hub. Growth shows no sign of abating, whether it occurs via Silicon Valley startups, or brick and mortar enterprises.
For infrastructure and operations teams, the growth of application containers is both good news and bad news. On the bright side, containers are a lightweight form of virtualization that make very effective use of underlying infrastructure -- container adopters report that the density with containers is more than ten times that of virtual machines running on a hypervisor. And because a lot of container implementations are open source, it's also a cost effective -- sometimes even free -- form of virtualization.
But containers' strengths can also be their weaknesses. Containers are small, stateless and ephemeral -- they can come online in seconds and disappear just as fast. Monitoring provider New Relic recently analyzed the data it collected as part of its new Docker Monitoring service, and found that the vast majority of Docker containers have a lifespan of less than five minutes. This sets us up to think of containers as a new class of servers.
"We're used to thinking about servers as pets and servers as cattle," said Abner Germanow, New Relic senior director of enterprise marketing. "Here, we have a new category -- servers as bacteria."
The container standards push
Efforts to standardize containers and their management have ramped up in recent months. In June, Docker announced that it would donate the code for its application format and runtime to the new Open Container Project (OCP) under the Linux Foundation, to avoid fragmentation. Notably, CoreOS, which had been promoting its own Rocket container format, signed on as a member of OCP.
Then, in July, Google announced that its Kubernetes project had achieved version 1.0 status, and then donated the code to the new Cloud Native Computing Foundation, also under the Linux Foundation. At launch, the foundation included 22 members, including Google, Docker, CoreOS and Mesosphere.
Containers are particularly copious when they are deployed as part of a microservices architecture, said Dustin Kirkland, Ubuntu product manager at Canonical, the sponsor of the Ubuntu Linux distribution, which supports Docker application containers as well as more traditional operating system containers. Using application containers in a microservices configuration, "the ethos is to put a single process in to the application container, and then orchestrate them in to a big complex service," he said.
But when you have a lot of single-process application containers, "things can get hard in a hurry," Kirkland said. "Dockerizing the first few applications is easy, but when you get to the point where you've Dockerized everything, that's when you get in to the realm of needing a container orchestration and management layer." As such, Kirkland prefers to think of containers as precious babies rather than expendable bacteria. "The smaller they are, the more attention, care and feeding they need."
The open-source community has been working on platforms and tools to manage and orchestrate dynamic microservices environments for years now, and those projects are gaining steam with the emergence of Docker application containers.
The Apache Mesos project, for example, is a distributed systems kernel build that works across a set of data center resources such as the compute, network, storage and applications, and arbitrates access to those resources.
The problem with the legacy data center is that "some human says 'I want to run on that machine,'" said Matt Trifiro, senior vice president with Mesosphere, which sells a commercial version of Mesos that it describes as a "data center operating system." Through a combination of elements such a resource scheduler and Linux init systems, Mesosphere abstracts the hardware and works to give applications "most of what they want" in terms of resources, Trifiro said.
CoreOS too, is working to extend the benefits of containers, with "Googled infrastructure for everyone else," said Alex Polvi, CoreOS CEO. Starting with a container and Google's Kubernetes container orchestration system, CoreOS Tectonic adds tools such as Fleet, a cluster management tool that presents the cluster as if it had a single init system, and Flannel, for container-to-container networking across disparate hosts.
"You need more than just Docker to build a useful system, the same way you need more than that a nail to build a house (even though your house is full of nails)," said Polvi.
Docker, meanwhile, has sought to capitalize upon its leadership position as the de facto format of the application container, and is developing a series of tools that can help with container orchestration and management. Examples include Docker Swarm, Docker Networking, and Docker Compose, all regrouped under the Project Orca initiative. And vendors that have traditionally offered platform as a service (PaaS) are working to increase their container management capabilities, including Red Hat, Deis and Jelastic, to name a few.
There are also closed-source approaches to the container management problem. StackEngine, for instance, recently released its Container Application Center, which provides configuration management, application deployment, orchestration and operations management, and is designed for use by both developers and operations teams. "Mesos and Kubernetes are complex and have a lot of building blocks," said Bob Quillin, StackEngine founder and CEO. "We solve the same problems, but coming from the enterprise."
Alex Barrett is editor in chief of Modern Infrastructure. Email her at firstname.lastname@example.org.