This content is part of the Essential Guide: The complete rundown on Docker data storage and containers

Top tools, techniques to scale container technology

Despite their many benefits, containers also present a challenge for some IT pros -- especially around scalability. Here are tools and tips to help.

Containers are all the rage these days, with both users and cloud providers rallying around the technology -- and for good reason. Among other benefits, containers offer a software architecture that provides portability and managed distribution of software assets. This has caused enterprises and software vendors to embrace container technology as they prepare for cloud.

But despite their success, containers still present challenges. Container scalability, for example, remains somewhat of a mystery. Some organizations struggle when trying to scale Docker, one of the leading container technologies. Luckily, there are certain processes and tools, including Docker Swarm and Google Kubernetes, that can help.

Choosing the right tool for container scalability

There are two basic approaches to scaling Containers. The first is to create a custom container management system that automatically launches new container instances, as needed, to handle an increasing processing load. However, since this is a DIY approach, the enterprise -- rather than a third-party provider -- is responsible for tasks such as system maintenance and updates.

The second way to scale containers is to use a third-party tool that handles container orchestration, scheduling and clustering. In general, these tools offer the basic mechanisms to enable scalability. But first, organizations need to pick the right container scalability tool for their needs.

Google Kubernetes is an open source container cluster management tool, much like Docker Swarm. With Kubernetes, organizations can schedule any number of container replicas across a group of node instances. Other tools use a similar container replication and distribution model, which is typically enough to scale most large, container-based applications.

Alternatively, Cloudify provides a Docker orchestration tool that overlaps with Docker Compose, a tool for running multicontainer applications, and Docker Swarm. Cloudify's tool has YAML-based blueprints that help developers describe complex topologies, including infrastructure, the middleware tier and application layers. Since it's more orchestration-oriented, organizations should consider Cloudify's tool when looking for orchestration and automation capabilities, not clustering.

A third option, and the newest tool, is Docker Swarm, which provides clustering, scheduling and integration capabilities. Using Swarm, developers can build and ship multicontainer or multihost distributed applications to scale and manage container-based systems.

Swarm competes with Kubernetes, which currently has a larger market share. However, if your organization needs to massively scale containers, consider both tools. Then, perform a proof of concept (POC) with each, using true-to-life workloads.

Four key steps to scale container technology

While tools like Kubernetes and Swarm help boost container scalability, organizations should also follow these four steps to more effectively scale their containers:

  1. Devote time to the architecture of your container-based applications. Poor designs, not poor technology, are the root of most scalability issues.
  2. Always do a POC to determine the true scalability of the each product you're considering. Use automated testing tools to simulate the workloads, as well as massive amounts of data for testing.
  3. Always consider your own requirements. What works for other companies may not be right for your container-based applications.
  4. Don't overlook container security and governance. Those technologies and processes have to scale as well.

Until the industry learns more about how container technology behaves at scale, container scalability will remain a challenge for some organizations. However, with a solid understanding of containers' proper use, and the right tools, you will be able to scale right out of the gate.

About the author:
David "Dave" S. Linthicum is senior vice president of Cloud Technology Partners and an internationally recognized cloud industry expert and thought leader. He is the author or co-author of 13 books on computing, including the best-selling
Enterprise Application Integration. Linthicum keynotes at many leading technology conferences on cloud computing, SOA, enterprise application integration and enterprise architecture.

Next Steps

Getting to know Docker container technology

Three apps that are a great fit for Docker

Five steps for deploying Docker containers

Why use Docker containers when VMs do the job?

Dig Deeper on IT Ops Implications of Continuous Delivery