alphaspirit - Fotolia

Blue chips share Chef DevOps adoption stories, best practices

How does an organization with a large legacy infrastructure Chef up DevOps? Liberty Mutual, Target and Capital One offer guidance and lessons learned.

AUSTIN, Texas -- There's an old saying about how to eat an elephant: one bite at a time.

Enterprises face a similarly daunting task in adopting DevOps into IT environments with legacy infrastructures. Several household-name companies spoke publicly here at ChefConf this week about the incremental ways they've adopted Chef DevOps practices and tools.

Taking things step by step has been key to the DevOps transformation at Liberty Mutual Insurance in Boston. The 100-year-old insurance company has teams in various stages of adopting DevOps and continuous delivery, said Mark Kirby, senior vice president and CTO of IT.

"If you're able to start with a blank piece of paper, you know all the things you'd do," Kirby said. "But if not, how can you get as close to that as possible?"

It's not quite as simple as going app by app or layer by layer of the infrastructure. "It's more like taking zones of things, value streams and integrating them," he said.

Liberty Mutual has internally developed applications to handle corporate department activities, such as purchasing approval, internal messaging and employee access to pay information. These applications are among the first to achieve a continuous integration and delivery (CI/CD) process.

Along the way, challenges included building in automation of auditing and compliance controls so developers can push code to production, as handoffs and the separation of duties required by some regulations "is counter to DevOps and Agile principles," Kirby said.

Packaged apps, however, are another story. "You actually have to have two DevOps transformation conversations," Kirby said. "One internally, and one with your vendors -- traditional enterprise vendors have to undergo the very same transformation."

Target's Chef DevOps transformation goes app by app

While packaged apps can be a challenge to integrate into a DevOps process, Minneapolis-based mega-retailer Target Corp. has begun to use Chef to build an immutable infrastructure for SharePoint.

You actually have to have two DevOps transformation conversations. One internally, and one with your vendors.
Mark Kirbysenior vice president and CTO of IT, Liberty Mutual

The process took six months of engineering effort and rationalization of whether it was feasible to rebuild immutable environments from scratch to accommodate changes, or to patch the existing environment, according to a ChefConf presentation by Naomi Reeves, senior engineer for Target.

"I'm no stranger to 'SharePoint sucks,'" Reeves said. But she also cited a comment by a Microsoft executive, saying 80% of Fortune 500 companies have SharePoint in on-premises production, to emphasize the application is a fact of life.

So, how to take an application that's complex and can be difficult to manage and make it part of a CI/CD process? That's where Chef comes in, although not without careful consideration of all options.

"Automation requires its own care and feeding," Reeves said.

Enterprises face a choice between abandoning legacy scripts that have already been used to automate parts of the environment in favor of a more systematic approach, or to spend time and energy trying to integrate those, as well as the application in question.

In Target's case, "we wiped the slate clean," Reeves said. Eventually, the team boiled the management of 19 SharePoint farms into one Chef DevOps cookbook.

"It's important to have small, reusable, attribute-driven recipes" within such a cookbook, Reeves said.

Since Chef allows infrastructure as code, developers were able to take over some of their own server provisioning within the environment. Reeves called it a big win for infrastructure operations at the company, which faced having to transform the SharePoint infrastructure while keeping up with daily operations tasks. Sharing the burden with developers helped free up the ops team to undergo the transformation project for SharePoint and other apps.

As a result, a process to build an immutable infrastructure for SharePoint -- originally estimated to require 214 days lead time and 19.5 days of engineering work -- was consolidated into a process requiring 14 days lead time and two days of work.

Capital One suggests Chef DevOps best practices

Capital One Financial Corp., the largest digital bank in the U.S., headquartered in McLean, Va., also presented here this week on ways large enterprises can incorporate Chef into existing automation and continuous delivery practices.

Like Target, Capital One faced the choice between discarding its legacy scripts and integrating them in a hybrid automation model, but found the latter approach more appealing, as it meant not abandoning its investment in developing the scripts.

In its quest to create similar delivery pipelines for applications and infrastructure alike, Capital One first had to train its employees to use tools such as Chef. This continues to require two-day training courses for all employees; junior employees are designated "sous chefs."

Capital One also needed a way to share the Chef Knife command-line tool with fine-grained access control, so only certain developers perform certain tasks, such as pushes to production and building an optional approval process for infrastructure changes, explained Ishu Gupta, cloud engineer for Capital One.

The company created its own process within the Jenkins continuous integration tool that allows for a consistent workflow for infrastructure as code, as well as application code. Capital One also built its own open source dashboard, called Hygieia, for auditability and loggability in the DevOps environment.

All the integration work was worth it, according to Surya Avirneni, senior software engineer for Capital One. Chef allows for automated unit test pipelines, as well as infrastructure pipelines that have consistent, repeatable results. It also saves time to automate testing of all features, with a focus on the recently released changes, rather than manually retesting features that might be years old, he said.

Discretion was also the greater part of valor in the DevOps adoption process -- the team chose to forgo add-ons, like Chef Search and Chef Delivery, or creating an end-to-end Chef DevOps process, for example.

Capital One found Chef Search is not ideal for service discovery in infrastructure clusters, because it doesn't take the health of systems into account when listing available resources.

As for Chef Delivery, "we had a strong pipeline already," said Ali Ravji, lead infrastructure engineer for Capital One. "Jenkins is doing the job."

Beth Pariseau is senior news writer for TechTarget's Data Center and Virtualization Media Group. Write to her at [email protected] or follow @PariseauTT on Twitter.

Next Steps

IT pros taste test Chef Automate offering

App automation comes home in Chef Habitat

What do you get with Chef Delivery?

Learn infrastructure as code template creation

Give Chef a try

Dig Deeper on Configuration Management and DevOps