This content is part of the Essential Guide: Use these DevOps examples to reimagine an IT organization

Crisis forges DevOps best practices at American Airlines, CSRA

There's no better time to institute DevOps best practices than during a broader disruption from a merger and spinoff, according to two large organizations, which tell their tales.

SAN FRANCISCO -- A period of upheaval following a company spinoff or merger may not seem like a great time to establish...

DevOps best practices, but for two large enterprises, such crises were exactly what drove their DevOps transformations.

When American Airlines acquired US Airways in December 2013, there was initially a lack of trust between the IT teams at each airline as the companies prepared to come together, said Susanna Brown, managing director of operations technology for Fort Worth, Texas-based American Airlines.

But "we just couldn't keep on throwing bodies at [the problem]," Brown said, speaking here at the DevOps Enterprise Summit (DOES). Instead, a new, more efficient way of working was required.

"DevOps ended up being our answer, and the merger became a catalyst for cultural change at American, which really allowed DevOps to take hold within our organization," she said.

Integration has been the top priority of the merging airlines, to move toward increased innovation once the value of the merger is realized. In early 2014, when integration work began, the newly combined organizations had 1,400 systems to bring together and about 2,000 projects to tackle. To date, the integration work is now 70% complete, Brown estimated.

Connecting teams was the foundation, and American Airlines set that stage in 2014 with team-building activities and collaboration technologies that united distributed IT teams in multiple geographies. Intense collaboration was introduced in 2015, and creation of new apps has been the culmination of the rest of the work in 2016. Along the way, the organization also standardized on two DevOps toolchains for Java and .NET workloads, which included configuration management with Puppet and dev and test automation with vRealize Automation from VMware.

More recently, Brown's airline technology group began cross-group collaboration with enterprise technology, which handles network, infrastructure and security for the airlines, as well as customer and corporate technology groups.

While American Airlines has mostly seen success on its integration journey, test automation remains an Achilles' heel of DevOps for the airline, Brown said.

"We recognize that this is an area where we're lagging, so we're trying to create lighthouse projects that we can use to showcase to the rest of the organization," she said.

The application that is furthest ahead in test automation is a flight attendant customer experience tool. Test automation reduced IT labor costs by 85% and testing time by 33%. And by the end of the year, the company expects a testing-time reduction of 66%.

Brown's organization has also created a self-service portal for developers to use to spin up infrastructure and encourage experimentation, though the portal has only begun to get internal traction, said Benjamin Chan, director of shared services for American Airlines, also speaking at DOES.

"You would think the developers would just run to this tool and be so excited, [but] there are always those cultural changes," Chan said. "We've released it; we've got some people on board. And as they're learning in these environments to try new things, or to experiment with software, to try algorithms, they're coming to us."

CSRA spins out, merges in with DevOps best practices

"A crisis is necessary for transformation," Paula Thrasher, director of digital services for CSRA Inc., a government IT services provider based in Falls Church, Va., said she has come to believe following the difficult birth of her company.

CSRA was formed following the spinoff of the North American federal government business belonging to systems integrator Computer Sciences Corp. (CSC) and a merger with Fairfax, Va., government IT service provider SRA International, all of which took place in a matter of months in 2015.

Prior to the CSC spinoff, other than some financial systems separated for auditing purposes for the public-sector business, IT was centralized between the government and commercial divisions of the business -- meaning the newly spun-off organization had to re-create an IT department from scratch.

"There was no IT department," Thrasher said. "What started off as a little bit of a shadow project, a network separation project under the auspices of security ... a few months later, it became public that we were going to split the company."

The announcement happened in April; the Securities and Exchange Commission filing date for the split was in October.

"We had to basically build an entirely new IT department, and we had about four months and one week to do it," Thrasher said. There was also no transitional period for CSC to make IT resources available to the new spinoff.

"To pull this off, we realized we had to break all the rules ... we're going to automate everything; it's going to be cloud. We're going to do all the things that you have to do to be fast, technology-wise," Thrasher said.

There was really no other option -- it was a once-in-a-lifetime chance to blow up IT and start over.
Paula Thrasherdirector of digital services, CSRA

To create a "minimum viable company," as Thrasher put it, the groups building the new IT department and the groups building out the new company's business functions were forced to collaborate from the beginning.

That stage of the separation process took shape around August 2015, when the team found out about the merger with SRA, which closed in November 2015.

"That was a really rough Monday," Thrasher remembered.

Still, at that time, the budding organization realized "anything other than DevOps was not success," she said. "This was our crisis. It forced us to transform, because there was really no other option -- it was a once-in-a-lifetime chance to blow up IT and start over."

To start, the team conducted what Thrasher called "the world's largest scream test ... that's where you unplug a system and you see who screams." This helped rationalize what applications and services to bring forward into the new organization. It also forced the business to tell IT what was truly important.

Along the way, apps were brought out of individual desktops, manually created spreadsheets, and legacy systems and into a cloud-based automated environment, including the automation of tests through Jenkins.

"Obviously, not everybody's going to get the chance to go through the joy of a merger and a separation," Thrasher said. "But the more you can bring the business into your DevOps journey, that's a huge success point."

Beth Pariseau is senior news writer for TechTarget's Data Center and Virtualization Media Group. Write to her at [email protected] or follow @PariseauTT on Twitter.

Next Steps

Here's how DevOps can be put to use

The effect of DevOps on a business

DevOps maturity isn't the same everywhere

Dig Deeper on DevOps Team Organization