.shock - Fotolia
BOSTON -- DevOps implementation has its challenges, even on the bleeding edge.
IT pros from six startup companies discussed their roadblocks to DevOps in a panel session at DevOpsDays here this week. One common issue is scaling small infrastructures and engineering teams into large ones, whether it involves technical scalability challenges or finding enough trained staff to handle the work.
"For us, the biggest challenge wasn't adopting DevOps so much as scaling it," said Vikram Pillai, chief architect and director of engineering at CloudHealth Technologies, a cloud service management platform maker in Boston.
With CloudHealth's first two or three engineers in a small office, communication was easy. As the company grew, communication among engineers wasn't as frequent, so it became harder to know what was happening in production.
"There were a lot of automated systems kicking in, and there were people doing specific stuff and they all stepped on each other," Pillai said. The solution was to reserve one of the company's 180 Slack communication channels for a streaming report on what's happening in production, he said.
Communication among geographically distributed DevOps teams is also a challenge for software as a service, or SaaS-based, infrastructure monitoring company LogicMonitor Inc., in Santa Barbara, Calif., which has three quarters of its developers based in China.
This means contending with a language barrier and culture shock among employees in China, where DevOps hasn't caught on yet, as well as navigating workdays on opposite sides of the clock, said Jeff Wozniak, operations engineer at LogicMonitor.
Meanwhile, scaling infrastructure for a DevOps implementation is just as tricky as scaling teams.
Figuring out when to move to a more robust cloud instance from a less expensive starter machine, or when to pull the trigger on taking something down that's working fine now, but won't next month, are tricky questions for Doug Rogers, director of product and co-founder of Pretty Instant Inc., an on-demand photography service in Boston.
"When to leap, I would say ... is the biggest challenge," he said.
Moreover, this scaling work must be done at the breakneck pace of business today.
"Project work used to be: We'll roll out a new Exchange platform; it's going to happen on this weekend; we're going to order the hardware four to six weeks ahead of time; it's scheduled on a Friday night, and everybody knows their downtime," said Glenn Grant, CEO of G2 Technology Group, a Boston-based company that does managed DevOps for small business.
"Now, those [changes] are coming through more rapidly and in smaller quantities."
DevOps implementation calls for hands-on training
DevOps implementations are nothing without skilled engineers at the helm, but education has not kept up with rapidly changing technology, panelists agreed.
"One thing I hope will happen is, as DevOps gets to be cooler and more people talk about it, maybe they can start introducing that concept in school," said Matt Williams, DevOps evangelist at Datadog Inc., which runs a SaaS-based monitoring platform in New York. Training in schools should go further than just writing applications, Williams said.
"OK, how are you going to deploy it? How are you going to get it out there in the real world?" he said.
Learning in the petri dish of a college environment is very different from learning on the job, and the latter is preferable, according to CloudHealth's Pillai. The company recently kicked off an internship program.
"We threw them in the fire," Pillai said of the interns. "They didn't do deployments, but they were writing code. How are we going to run it, how are we going to know if it doesn't work, and how are you raising failures?"
The prospect of DevOps certifications was raised during a Q&A with the panel, but most panelists said certifications lack real meaning compared with job experience and the ability to solve practical problems.
"I'd rather someone made a cool thing than took a [certification] test," Rogers said." I'd rather hear, 'Check out what this does, that took me a weekend.' That speaks to me more in terms of the value to the team."
One session attendee noted that DevOps certifications at this early stage of the movement could put an emphasis on tools, rather than culture.
"You can't get a certification in sending concise, short, well-composed email [messages] to the right people," he said.
Three DevOps challenges you will face
Where DevOps goes wrong
The toughest ops challenge of DevOps
Quality, stability and velocity are the secrets to DevOps success