This content is part of the Essential Guide: Guide to building a better IT team structure

The bloom is off the DevOps rose

There's a lot of hype about continuous deployment in DevOps circles, but it can be disastrous if the right culture and organization aren't in place.

NEW YORK -- The bloom is off the DevOps rose, if ever so faintly.

The rush by some organizations to implement a DevOps mainstay, continuous deployment (CD), was skewered by Avishai Ish-Shalom, CTO at Fewbytes Technologies, an Israeli IT consulting firm specializing in cloud systems and automation. Ish-Shalom slammed organizations for deploying continuous integration (CI) and CD tools without the proper organizational structure in place, or for the wrong reasons, in a session here at O'Reilly's Velocity Conference.

Implementing CD (a close cousin of another CD, continuous delivery) can speed up the rate at which new code is delivered, first with quality assurance testing, encouraging teams to automate servers and builds, and eventually deployment. Things are going swimmingly, "until the system breaks, and nobody can get it back up," Ish-Shalom said.

"With CD, the processes that you put in place to help you introduce changes break your system more often, in a more disastrous way than before," Ish-Shalom said. "There's a lot of hype around continuous deployment, but no one is talking about the organizational and cultural implications," he said. "At the office, we call it continuous disaster."

For one attendee, the case against CD is more pragmatic: compliance. Providing both devs and ops teams with access to test and production systems can be a direct violation of some security regulations, for example Sarbanes-Oxley Act, said Chris Maier, principal architect for product shared services with managed cloud provider Rackspace. In such environments, developers are not allowed to have access to production systems, and ops teams don't have access to test and dev environments, in order to enforce a separation of concerns.

DevOps is still a very worthy goal, but just implementing the right tools and calling yourself DevOps doesn't cut it, said Rodrigo Campos, IT operations manager for Brazil.

"We had Chef, OpenStack, Docker and Git, so people would say, 'that makes us a DevOps shop, right?'"  Campos said. In truth, "things were a mess, even though we had a DevOps team. The operations team dealt with legacy infrastructure, and the DevOps team was filled with superstar ninjas," he said, straining relations between the groups.

Campos said actually saw better results after it eliminated its dedicated DevOps team and replaced it with production management and production engineering groups that interface with the business and infrastructure teams, respectively, and where everyone takes ownership and responsibility for the success of the web site.

To get to DevOps, "we needed to change the mentality," Campos said. More than tools or even processes, Campos said his definition of a successful DevOps organization is one that is communicative, collaborative and transparent.

"That's it."

Next Steps

Implement a solid DevOps team structure

Adopt a successful DevOps enterprise

Dig Deeper on Scripting, Scheduling and IT Orchestration

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

DevOps is yet another in a long line of so-called "new" ideas meant to address problems that would not exist if organizations simply executed fully and competently on an existing and proven best practice. In the case of DevOps, the actual solution is to properly implement their Architecture Practice. Done properly, the competently-executed combination of Enterprise, Portfolio, and Solution Architecture fully address those needs with time-proven approaches. .
The idea that responsibility for performance and security need to be intertwined is exactly right. Working in the DevOps space, one of the trends I often see is that regulatory requirements, whether from SOX or HIPAA or PCI-DSS, have specific implications for teams that are using some of the tools you mention. There needs to be a separation of privileges in order to embrace DevOps without falling afoul of the regulators; things like SSH private keys should never be stored in GitHub, for example. CD isn't a disaster, but the technologies around authorization and zero-trust need to catch up -- too many companies pull the trigger on the idea of DevOps without thinking through how they will manage the daily workflows in a way that is compatible with their business (and audit) requirements.
DevOps is first and foremost a culture. After that comes methodologies and tools. If yuo jsut have a bunch of tools all you have is a bunch of tools. And if your Ops team and Dev team are not working as "one" then you are not even close to the culture. As to compliance that is about methodology. Nowhere does DevOps says that the responsibilities across environments are shared. That is an organizational decision and has to be made in the context of what is appropriate for your business.
I echo @BeenThereDoneThat99. This is not new, and Architects have been wrangling exactly these problems for decades. The buy a tool to solve all ills argument is well worn, alone it is unlikely to achieve anything useful. However, since there appears to be a resistance to simply implementing "Architecture Practice" one has to surmise that the organisation is not functioning correctly. So the communication aspect of DevOps (if that's how we wish to term the coupling of methods, tooling and culture) becomes ever more important. Individual engineering teams cannot do this alone and the guys that need to be at the forefront of this are the Architecture teams. That is employing scientific method to making changes to our value generation output. Hypothesize, design, implement, test and refine in an industrialised way. Until we revert to this way of working we will continue to experience some of the chaos that has led to this article.