adam121 - Fotolia


Look inside the Mesosphere DC/OS distributed operating system

IT organizations that run large, distributed workloads on clusters should examine the Mesosphere DC/OS approach, but be forewarned: It takes a lot of components to simplify ops.

While Mesosphere DC/OS can ease the management of complex, containerized architectures, IT admins have a lot to learn about the components of the OS before adoption.

The Mesosphere Data Center Operating System (DC/OS) is a distributed OS that runs across a cluster. It's based on the open source kernel Mesos -- similar to the Linux kernel -- which abstracts IT hosting resources. Mesos uses a master/agent architecture, whereby a master daemon controls the agent daemons that run on nodes within the cluster.

Because DC/OS is a distributed system, it lends itself to distributed applications.

For example, an enterprise could use Mesosphere and its distributed architecture to run large and complex data processing software for a big data initiative, such as Apache Cassandra database or Apache Spark data analytics engine, across abstracted virtual and physical resources that act as a single entity. Mesosphere DC/OS enables greater scalability for those systems, since the OS spans more than one machine, and also simplifies node configuration across a cluster.

Core DC/OS functions

When used alongside Marathon container orchestration, DC/OS acts as a cluster management and container orchestration system, with its primary function being to push containers out to nodes. Mesosphere DC/OS can work with Marathon or the more widely adopted Kubernetes container orchestrator.

DC/OS performs other IT operations tasks, so users can provision storage, build out networks, monitor performance, collect logs, manage configurations and control identity. It also performs service discovery via Mesos DNS, and the distributed systems management server Apache Zookeeper provides a high-availability mode, in which configuration information is replicated from a leader master node to follower master nodes.

Mesos DNS, Zookeeper and Marathon are just a few of the many components that enable Mesosphere DC/OS, as shown in the full diagram of components in figure 1.

Mesosphere DC/OS components
Figure 1. Mesosphere DC/OS relies on a complex mix of components to support cluster and container management, networking, security and other IT ops tasks on master and agent nodes.

Container architecture and catalog

Even though the Mesos kernel abstracts Mesosphere DC/OS, it does not run on bare metal. Each node must have an OS or VM installed, plus the Docker container software. All Mesosphere tasks -- which are application instances -- run in containers. DC/OS can pull in pre-packaged containers from repositories, such as Docker Hub and the Google Container Registry.

DC/OS uses Mesos for cluster management, Marathon for orchestration and DC/OS Jobs to schedule tasks -- all of which combine to drive container orchestration processes. DC/OS supports Mesos and Docker container types, and also includes a Kubernetes-as-a-service option for container orchestration.

Since DC/OS acts as a container orchestration system that uses Docker, its catalog resembles the catalog of containers users can download from Docker Hub, as seen in figure 2.

Mesosphere DC/OS catalog
Figure 2. The Mesosphere DC/OS catalog includes certified packages for users.

Mesosphere DC/OS installation requirements

Mesosphere DC/OS has a number of hardware and software requirements. A main component of the installation process, the universal installer, can run on Mac, Windows and most versions of Linux.

The master node requires 32 GB of memory and four processor cores. The bootstrap server, used to start the installation, requires 16 GB, as do the agent nodes. Nodes are either private, with an internal IP address, or public, with a public IP address reachable from the internet.

DC/OS runs on CentOS, Red Hat Enterprise Linux, CoreOS or Oracle Linux. It can also operate on public cloud platforms, including Microsoft Azure, AWS and Google Cloud.

The Admin Router

Admins can work with Mesosphere DC/OS via its web-based GUI, the command-line interface and an API gateway, called the Admin Router. The gateway acts as a proxy server, based on Nginx, that forwards the requests it receives to agents.

The API natively supports the Lua programming language. Since the gateway is web-based and has a transport layer that uses HTTP/HTTPs, admins can use any programming language -- but via a public HTTP protocol, as opposed to a native DC/OS connection. For example, a URL is formatted as http://<cluster-url>/<route>/<file-path>. If admins use Lua, they do not need to add a URL to API requests.

Figure 3 shows how the Admin Router load balances traffic from the network.

Mesosphere DC/OS Admin Router
Figure 3. Mesosphere DC/OS users can work via a GUI or CLI, as well as the Admin Router, an API gateway.

Dig Deeper on Managing Virtual Containers