alphaspirit - Fotolia

Chef automation branches out with new Automate product

Chef CTO Adam Jacob explains how the company's new Chef Automate product gives IT pros insights across infrastructure automation, application automation, security and compliance.

AUSTIN, Texas -- Chef isn't standing still as the fast-paced DevOps world evolves.

Adam Jacob, CTO and co-founder of Chef Software Inc., based in Seattle, was the original author of the configuration management tool, and has recently reinvented it with a product called Chef Habitat. Now, Chef Habitat is being linked with an updated version of the Chef Delivery pipeline, forming a new commercial offering, called Chef Automate.

We caught up with Jacob for a sneak peek at the Chef automation capabilities wrapped up in the new product, which has been kept carefully under wraps for months and will be demonstrated on the keynote stage this week at ChefConf.

First, what is Chef Habitat?

Adam Jacob, CTO, ChefAdam Jacob

Adam Jacob: Chef Habitat is what we call application automation. What we mean by that is, historically, when we think about automation, we think about it being around the application. It's infrastructure that you build, or it's systems that manipulate the application, but the application pays no mind to any of those things. If you think about the lifecycle that applications go through, there's writing the application itself; then, there's building the application, getting the artifact; then, there's deploying that artifact into an environment; and then there's managing it over its life.

Habitat flips the table over and says, 'What would happen if we built automation from the application's point of view? What if the application was responsible for doing all of that build/deploy/manage loop, and how would that work?' The big aha moment was that the automation has to travel with the application. So, rather than having the application travel, and then having the automation travel in parallel, or having lots of different infrastructure systems that provide services to an application that are about that build/deploy/management cycle, Habitat says that everything the application needs to do to build, deploy and manage itself happens with the application artifact. When you build the artifact, all the automation is there as well, and when you deploy it, it has everything you need to manage it through its life. And that includes complicated stuff, like topologies.

What's a topology in this context?

Jacob: A topology is how a service is structured. For example, you might have a database, and you might have a write leader and read followers. That's a topology -- it's inside a service, and it's how they're configured to each other. And then there's service discovery, which is, I need to link from one service to another service. How do we know where they are? And how do we know how to contact each other? And how do we know credentials? How does the load balancer know what web servers to load balance? There are things like updates, so when a new version of software is available, how does the service automatically update itself with the new version without taking everything down?

All of those things Habitat does for you as a part of building the application. We do that by taking a lot of the data that you need to manage that lifecycle and include that in the build artifact, and then when you run a service with Habitat, there's a supervisor that runs the services you define in those packages. The supervisor can connect to other supervisors and form a ring, and they share information about the services they're running within the ring, and they can use that information for service discovery and topologies. So, it replaces a bunch of the infrastructure you would have to deploy without Habitat, and it makes them really easy to reason about because they travel with the app.

Rather than saying, 'Before we deploy these services in production, we need to deploy these five other services,' with Habitat, you just deploy the application and connect them together, and then the ring does the rest.

What does Chef Automate do that Chef Habitat and Chef Delivery didn't do on their own?

Jacob: The workflow feature of Automate used to be Delivery. What Delivery gave you was a workflow that is good at enabling teams to ship software at high velocity. It gives you that continuous delivery pipeline. When we look at what's difficult about adopting it, we lowered the barrier to teams to do continuous delivery by providing a system with user-serviceable parts that offers what you need, but nothing more. We removed that conversation about the shape of the pipeline and how stuff flows, and do you do code reviews or don't you.

The layer where it was still very hard was, you have an application team -- how does that application get built? How does the artifact work? Where do we store the artifact? How do we know which versions to deploy and where they deploy into? How do we orchestrate the update strategy of a new version of software rolling into an environment?  All of that -- the pipeline shape I can show you, but you still have to go to these teams and you have to implement all the answers to all those questions. Habitat gives you answers to all of those questions in a really simple and easy way. It's way easier to get applications into a continuous delivery pipeline now, because they behave correctly.

If you're building Chef automation into the application, what does IT operations do?

Jacob: They do what they have always done. There is infrastructure required to run those applications, and that infrastructure needs to be managed and maintained. And application teams need operations people to help them understand how to choose the right one, how to interact with it, how to troubleshoot it and how to tune it. While the automation travels with the application, Habitat itself forms a giant distributed application, so the supervisors themselves become this giant distributed system, and that thing itself needs to be understood, managed and maintained.

People in that operations role need to see what's happening across their workflow and across their infrastructure automation, application automation, and security and compliance posture, and [they] have to see all that in the same place.
Adam Jacobco-founder and CTO, Chef

What we learned with Automate was that the separation we had between Delivery, Habitat and other Chef products made it difficult to visualize everything that was happening. People in that operations role need to see what's happening across their workflow and across their infrastructure automation, application automation, and security and compliance posture, and you have to see all that in the same place. We had them separate [with Chef automation before], and that was a mistake. So, now, they're no longer separate; now, they're one system that takes all the data from all three of those places, pulls them together and shows it to you all at once.

So, you're opening up a lot of new fields of competition for your product. I'm thinking of HashiCorp's Vault, for example.

Jacob: I don't want to compete with Vault, in that what I did with Habitat wasn't to make a standalone security solution you can deploy to configure secrets. What I did was build an application automation platform that enables applications to run efficiently. In order to do that, I also needed to manage their secrets.

Vault can be a really impressive and effective certificate authority for [Secure Sockets Layer] SSL -- [something] Habitat's not and never will be. What it does is allow you to securely distribute and encrypt that SSL certificate for a particular service endpoint, and be sure that it gets distributed every time you launch a new instance of that service, because that's the application's job. It does that in the same way it does service discovery, but it doesn't replace etcd or Consul, because those are generic systems that you can use to build other generic systems on top of, and Habitat is not. But there are things you can do with those platforms that you might not do because you're using Habitat, and you don't need those [features] anymore.

Does it open up new fields of competition? I guess it does, but it's designed from a different point of view.

Beth Pariseau is senior news writer for TechTarget's Data Center and Virtualization Media Group. Write to her at [email protected] or follow @PariseauTT on Twitter.

Next Steps

Go farther with cloud-based DevOps

Lessons from Facebook's Chef implementation

Chef's regulatory compliance feature set

Dig Deeper on Configuration Management and DevOps