Les Cunliffe - Fotolia

Kubernetes on AWS still a work in progress as management tools emerge

Kubernetes and AWS are hailed as the leaders in their respective markets -- so why don't they work more closely together?

Kubernetes management on AWS has lagged behind that of other popular public clouds, but that may be about to change.

Kubernetes Operations (kops) is a utility that eases the spin-up of Kubernetes clusters on the Amazon Web Services (AWS) cloud. Such environments in the AWS public cloud still await production-ready Kubernetes 1.7 support, as kops support for 1.7 remains in beta. Kubernetes kops is not required to run Kubernetes on AWS, but it holds strong appeal for a user audience that has no other formal Kubernetes support native to AWS.

The divide between Kubernetes and AWS -- which arguably form the most popular container orchestration and public cloud platform combination -- involves politics on both sides of the fence, analysts said.

Amazon has dabbled in Kubernetes support but has yet to uphold the Kubernetes scheduling engine within Amazon Elastic Compute Cloud Container Service. On the other side, the Kubernetes community may be wary to engage the public cloud juggernaut, said Fintan Ryan, analyst at RedMonk, based in Portland, Maine.

"Customers absolutely want more integration," Ryan said. "There's a feeling in the Kubernetes community that they can manage all the layers and they don't want to get locked in with Amazon."

Still, the writing is on the wall. AWS has increased its participation in the community recently and has attended recent meetings of the Kubernetes AWS Special Interest Group (SIG-AWS). One engineer who is an active member wrote an official AWS blog post on best practices for using Kubernetes on AWS with kops. AWS also has begun to embrace the development of upstream Kubernetes as part of its expanded partnership with Red Hat on OpenShift, which kicked off in May 2017.

"The market will speak -- AWS is where a lot of people want to run Kubernetes," Ryan said.

Kubernetes kops development lags behind main codebase

One shop looking to run Kubernetes on AWS is Kairos AR Inc., a provider of human facial recognition and analytics algorithms for developers in Miami. Kairos has put together a Kubernetes management environment in AWS based on HashiCorp's Terraform infrastructure as code tool, Red Hat's Ansible for configuration management, Calico for software-defined networking, and some of its own custom scripts, but the team wants to switch to kops soon.

"The setup that we have is kind of brittle," said Cole Calistra, CTO of Kairos. "Every time we update Kubernetes, it's painful because we have to make sure all the dependencies and versions of Docker, Kubernetes and Calico are correct. But using something like kops, all that stuff is taken care of for you."

There's a feeling in the Kubernetes community that they can manage all the layers and they don't want to get locked in with Amazon.
Fintan Ryananalyst, RedMonk

In less than a month since the Kubernetes 1.7 release, kops integration has already reached beta status. Nevertheless, "my experience with Kubernetes tells me to wait until the [version 1.7.1 or 1.7.2] release and avoid the alphas and betas if you can," Calistra said.

In the meantime, it's a bit of a chicken-and-egg situation as Calistra's team awaits stable Kubernetes 1.7 support in kops before it updates its clusters to the latest release of the container orchestration platform.

"Once we're switched to kops, updating the cluster to a new version is not going to be as big a deal anymore," Calistra said. "It won't be a big deal to spin up a new cluster just to test it; whereas, now, I have one large cluster that wasn't really written with running multiple environments in mind."

Even with kops, Kubernetes on AWS management could still be better and has some catching up to do with Google Container Engine, Calistra added. For example, while the open source AWS Blox software can be used to integrate AWS with the Kubernetes scheduler, there's no production-ready control plane for multiple clusters that span different regions in AWS in kops.

Kubernetes can also restrict the amount of memory and CPU a pod can use within a cluster, but those limits should be determined without guesswork, he said. AWS requires an overlay networking provider, such as Calico. The built-in Kubernetes network utility, Flannel, sometimes bumps up against AWS limitations on the size of routing tables.

Kubernetes kops progress -- along with the development of kubeadm, a utility meant to bring the advantages of kops to a wider array of Kubernetes environments outside of AWS -- has been too slow for some community members. Kubeadm overall doesn't integrate with AWS as well as it should, though it was ready to take advantage of Kubernetes 1.7 multi-tenant security features right away, while kops won't iron out that support until at least version 1.7.1. The AWS Quick Start for Kubernetes, based on kubeadm, will also have to wait at least until the end of the month for a brush-up on the way the kubelet matches names in the API server with its counterparts in the AWS network.

"We're getting better with the Kubernetes releases, but they're still a little rough," said Joe Beda, one of the original authors of Kubernetes at Google who now serves as CTO of Heptio, a Kubernetes development startup created by ex-Google IT pros in Seattle. "Kubernetes 1.7 is more buttoned-up than previous releases, but there is still some fit and finish stuff that lags a little bit behind."

Beth Pariseau is senior news writer for TechTarget's Data Center and Virtualization Media Group. Write to her at [email protected] or follow @PariseauTT on Twitter.

Next Steps

Open source community seizes Kubernetes usability opportunities

Map containers to infrastructure with Kubernetes Nodes and Pods

What the OS has to say about containerization

Dig Deeper on Managing Cloud-Native Applications