Enterprises federate container cluster technologies to unify and manage multiple domains, while keeping them autonomous. Federation, meaning to combine without homogenizing, creates a hybrid cloud setup with separate lifecycles and deployments. It expands the scope of container deployment across hosting options, but it's tricky to do successfully.
There are two dominant approaches to container federation. The first approach is the Apache Mesos and Marathon combination. The second is Kubernetes, either as a stand-alone platform or used with other tools.
Mesos approaches federation as a peer group of equal parties that cooperate, while Kubernetes federation has a more specific master-worker relationship, where the master unites the operators to support the common goal. Enterprises should compare Mesos vs. Kubernetes based on their specific container goals.
Mesos for abstraction
Mesos is a platform that accommodates large-scale container deployments. It enables multiple distributed data centers to act as a common container hosting pool. Mesos operates alone, or in conjunction with DC/OS for increased simplicity and ease of use. [Editor’s note: Mesosphere – the company that maintains DC/OS -- relaunched as D2iQ in November 2019.] The Mesos kernel supplies most of the federation features, whether you choose the enterprise distribution or open source framework. Marathon is the orchestrator associated most often with Mesos, but users can alternatively orchestrate a Mesos deployment using Kubernetes.
This article is part of
Mesos offers two different models of federation. One approach puts all infrastructure beneath a control layer and abstracts resources from a data center, public cloud, VM deployment or other source; it can be thought of as submergence. The control layer presents a uniform abstraction for hosting from above. In the second model, the technology collects separate Mesos deployments in such a way that none are subordinate, but all cooperate in a totally distributed and fault-tolerant manner.
Kubernetes federation options
Kubernetes is an orchestrator that dictates where containers run on hosting resources. Containers are deployed in Kubernetes pods on bare-metal or VMs, and the latter enables IT teams to integrate public cloud IaaS offerings with each other, or with data center container hosting. A local operator process, controlled under a master process, manages the deployments -- wherever they are hosted -- beneath a Kubernetes control plane that passes along all the deployment instructions and feedback.
Kubernetes federation is a feature in flux. The original federation model, called version 1.0, is obsolete, and Kubernetes documentation tells users to adopt version 2.0. Red Hat OpenShift is an example of a commercial Kubernetes distribution that uses version 2.0's federation model. Both the obsolete and emerging approaches focus on multi-cluster federation.
Basic Kubernetes users can add tools from the orchestrator's ecosystem to extend the control plane to multiple clusters. NetApp's Stackpoint, Rancher and Juniper Network's HTBASE are a few ecosystem options for Kubernetes federation. Service mesh network technology, such as Istio or Nginx, are options to create connectivity and add load balancing to federated deployments.
All the tools to extend Kubernetes' control plane provide for redundant hosting of the master element, but there is still an explicit master-operator relationship, unlike federation with Mesos. The Kubernetes approach is best suited to deployments where the individual Kubernetes clusters are largely autonomous, but sometimes cooperative.
Choose between Mesos and Kubernetes
Mesos vs. Kubernetes comes down to established vs. the up-and-comer.
Mesos has many positives for container federation. It is a multifeatured technology that supports container deployments that run everywhere, across tens of thousands of hosts, with hundreds of thousands of containers. Mesos dominates for large-scale container operations. But Mesos' rich feature set makes it more complicated than Kubernetes, and many enterprises won't use all those features.
None of the Kubernetes federation options are proven at a similar scale to those of Mesos, yet there are more Kubernetes users than there are Mesos users. That broad user base puts Kubernetes at the center of a growing ecosystem, and some Mesos users are evaluating a Kubernetes migration.
The Kubernetes ecosystem enables capabilities, such as resource abstraction, that were once the domain of Mesos. Paired with HashiCorp's declarative infrastructure-as-code tool, Terraform, Kubernetes creates a resource configuration framework that enables a universal resource model for orchestration to work through. The coordination with Terraform gives Kubernetes the agility to manage almost any resource in any cloud.
This ecosystem is decisively shifting the container federation landscape in favor of Kubernetes over Mesos. It combines Kubernetes federation with tools that facilitate monitoring and lifecycle management, as well as service discovery and deployment.
Kubernetes federation version 2.0, or Kubernetes in combination with appropriate ecosystem tools, is the right approach for two main enterprise IT scenarios:
- organizations with a mix of many applications built from a fairly conventional number of components, as opposed to a huge array of them, that require load balancing for only a few front-end elements; and
- those that use managed Kubernetes services from various cloud providers.
Mesos federation remains the best choice for some enterprise container deployments. For example, Mesos is the right approach for a few applications, each made up of many components, globally distributed and load-balanced across the entire scope of enterprise IT infrastructure. Mesos appeals to web giants, which tend to run this kind of deployment. But most enterprises are better off with Kubernetes and appropriate supporting tools.