Sergiy Serdyuk - Fotolia
A digital advertising firm found a balance between the ease of a fully managed service and the control of consultative enterprise support in a Kubernetes management startup as it moved away from Mesosphere two years ago.
Zeeto, a digital ad network in San Diego, had used Mesosphere's DC/OS for container orchestration as Docker first rose to popularity in 2016 and 2017, but by November 2018, market momentum had begun to shift toward Kubernetes. As a firm with a 15-member staff of engineers tasked with everything from product development to infrastructure management, Zeeto needed help moving to a new container management platform, and couldn't afford the support cost of enterprise platforms such as Mesosphere DC/OS at an estimated $2,000 per node, per year.
"A do-it-yourself approach was not really in the cards based on the staffing we had," said Matt Ferguson, CTO at Zeeto. "All of my engineers also handle DevOps and support, and the migration looked pretty monumental, considering all the other things that were already priorities."
However, a fully managed Kubernetes service, whether from a cloud provider such as Google or a SaaS vendor, also wasn't a good fit for the firm, which wanted to maintain control over the infrastructure. And an enterprise software platform such as Red Hat's OpenShift would have been too expensive.
Instead, Ferguson met the CTO of a startup, Agile Stacks, at a San Diego industry group, and decided to become an early adopter of the firm's software and services for Kubernetes management and DevOps pipeline integration with the container platform, despite the fact that the company and product were in their early stages.
"There is more to Agile Stacks than just being Kubernetes experts," Ferguson said. "They also are DevOps experts, so they can support us in building our apps, our pipeline, our build tooling, to deploy properly into that environment."
Kubernetes management support a matter of trust
Agile Stacks, which completed a $6 million Series A funding round in July 2019, remains a relatively small, emerging company, compared to Kubernetes management software firms such as IBM/Red Hat, Rancher, VMware Heptio and D2iQ, as the company formerly called Mesosphere is now known.
Many of these companies, particularly D2iQ, offer enterprise support, training and consulting services for Kubernetes at a wider variety of price points than were available in 2018 with platforms that targeted large enterprise early adopters, such as Mesosphere DC/OS. D2iQ declined to respond to requests for current pricing but uses a freemium model for its Kubernetes distribution, Konvoy.
Agile Stacks also has other startup competitors, such as managed Kubernetes firm Fairwinds, which released a user-managed distribution of Kubernetes alongside user-managed monitoring and security tools last year, as well as consulting firms such as Altoros.
This market environment gives smaller businesses such as Zeeto more options for container support, albeit with some caveats.
Matt FergusonCTO, Zeeto
Agile Stacks is a new, small vendor, which carries some risks, Ferguson acknowledged, though his team has become much more confident with Kubernetes skills and could take back control if necessary. But Zeeto has also grown used to Agile Stacks support for Kubernetes upgrades and monitoring, which would present a challenge in the event Agile Stacks goes out of business or Zeeto wants to end its relationship with the vendor.
Agile Stacks' relative size also makes it able to respond quickly and work closely with engineers at firms such as Zeeto, which might not get the same attention from a larger vendor. And Zeeto's relationship with Agile Stacks amounts to a kind of hybrid between a service provider and a software vendor. So far, Zeeto has chosen carefully what it wants to outsource to a vendor and where it wants to maintain full engineering control, rather than buy in to Agile Stacks' full product line.
"There's a lot more that Agile Stacks could probably do for us than we initially let them do, because we did have a pretty sophisticated pipeline already," Ferguson said. For example, Agile Stacks can deploy Jenkins X CI/CD on Kubernetes, which Zeeto's team is considering, but hasn't decided to embrace yet. Agile Stacks' Kubernetes management packages for ephemeral machine learning resources also aren't a good fit, since Zeeto trains its machine learning algorithms daily and thus requires persistent resources for such workloads.
There is also some risk inherent in handing over Kubernetes management to a third party.
Still, to Ferguson, this is preferable to a fully managed SaaS Kubernetes service.
"If I don't pay them, or I decide to move on, I still have their scripts running -- I still have the baseline of what we know to be Kubernetes," he said. "I would have to go it alone and figure out how to maintain that myself, and I'm not going to have their updates and source code anymore, but I would still have a system running on my own infrastructure on AWS."
Kubernetes management support beyond containers
As Zeeto moved from Mesosphere to Kubernetes on AWS, Agile Stacks provided some prepackaged scripts for deploying containerized apps onto the container platform via Jenkins, connected with HashiCorp Vault for secrets management, and used Terraform and AWS CloudFormation for infrastructure as code (IaC) deployment. Agile Stacks also provides monitoring software and services along with managed upgrades for Kubernetes, but its support went beyond a typical Kubernetes managed service, as it also consulted on issues outside the container platform.
For example, Agile Stacks engineers and scripts helped Zeeto deploy the Istio service mesh and troubleshoot its notoriously finicky setup for blue/green and canary application deployments. The company also helped Zeeto resolve complicated root cause analysis for a compatibility problem between MongoDB, Kubernetes and AWS Linux nodes that caused MongoDB to lose connectivity to Zeeto's apps every few days. Finally, Agile Stacks helped set up Zeeto's Kubernetes environment on AWS Spot Instances, instead of Reserved and On-Demand Instances, which cut cloud infrastructure costs.
"We were able to launch on Spot Instances with their assistance without becoming experts," Ferguson said. "Now we have a lot more sophistication that we can bring to the table, but when we were first launching this, we definitely couldn't have done it as quickly -- it would've taken us six months to get it right."
Zeeto pays about $2,000 a month for this broad level of support, about the same as it would have paid per node, per year just for Mesosphere DC/OS support, Ferguson said.
As Agile Stacks offloaded some of the firm's Kubernetes management tasks, Zeeto's engineers focused on product development, including a new AI-driven polling system for market research it plans to launch this year.
"If we'd had half the team focused on standing up Kubernetes and making it bulletproof, we would have been six months behind that schedule, which means we wouldn't be hitting this political cycle with our polling product," Ferguson said. "We didn't know [in 2018] we'd be going into polling, but that turned out to be an important revenue source for the company that we might have missed."