Enterprises have a new weapon to combat the IT skills shortage where new hiring and training practices fall sh...
Most IT pros agree the fastest path to IT burnout is what Amazon engineers have termed "undifferentiated heavy lifting," which is repetitive and uninteresting work that has little potential for wider impact beyond keeping the lights on. DevOps tools training, which involves IT automation practices, can reduce or eliminate such mundane work and can compensate against staff shortages and employee attrition.
"Automation tools aren't used to eliminate staff; they're used to help existing staff perform at a higher level," said Pete Wirfs, a programmer specialist at SAIF Corp., a not-for-profit workers' compensation insurance company in Salem, Ore., that has used Automic Software's Automation Engine to orchestrate scripts.
The company has used Automation Engine since 2013, but last year, it calculated new application development would add hundreds of individual workflows to the IT operations workload. Instead, Wirfs said he found a way to automate database queries and use the results to kick off scripts, so a single centralized workflow could meet all the project's needs.
As a result, SAIF has expanded its IT environment exponentially over the last four years with no additional operations staff. The data center also can run lights-out for a few hours each night, with the automation scripts set up to handle monitoring, health checks and route alerts to the appropriate contacts when necessary. No IT ops employees work on Sundays at SAIF at all.
"There's no end to what we can find to automate," Wirfs said.
DevOps tools training standardizes IT processes
SAIF's case illustrates an important facet of DevOps tools training: standardization of a company's tools and workflows. A move from monoliths to microservices can make an overall system more complex, but individual components become similar, repeatable units that are easier to understand, maintain and troubleshoot.
"The monoliths of the early 2000s were very complicated, but now, people are a lot more pragmatic," said Nuno Pereira, CTO of iJET International, a risk management company in Annapolis, Md. "DevOps has given us a way to keep component complexity in check."
In modern monitoring systems, DevOps tools training can curtail the notifications that bombard IT operations pros through centralized tools, such as Cisco's AppDynamics and LogicMonitor. These are popular among DevOps shops because they boost the signal-to-noise ratio of highly instrumented and automated environments, and they establish a standardized common ground for collaborative troubleshooting.
"[With] LogicMonitor, [we can] capture data and make it easily viewable so that different disciplines of IT can speak the same language across skill sets," said Andy Domeier, director of technology operations at SPS Commerce, a communications network for supply chain and logistics businesses based in Minneapolis.
Four or five years ago, problems in the production infrastructure weren't positively identified for an average of about 30 minutes per incident, Domeier said. Now, within one to two minutes, DevOps personnel can determine there is a problem, with an average recovery time of 10 to 15 minutes, he estimated.
Standardization has been key to keeping up with ever-bigger web-scale infrastructure at DevOps bellwethers such as Google.
"If every group in a company has a different set of technologies, it is impossible to make organizationwide changes that lift all boats," said Ben Sigelman, who built Dapper, a distributed tracing utility Google uses to monitor distributed systems. Google maintains one giant source-code repository, for example, which means any improvement immediately benefits the entire Google codebase.
"Lack of standardization is an impediment to DevOps, more than anything else," Sigelman said.
Google has standardized on open source tools, which offer common platforms that can be used and developed by multiple companies, and this creates another force-multiplier for the industry. Sigelman, now CEO of a stealth startup called LightStep, said DevOps tools training has started to have a similar effect in the mainstream enterprise.
Will AI help?
DevOps tools training can go a long way to help small IT teams manage big workloads, but today's efficiency improvements have their limits. Already, some tools, such as Splunk Insights, use adaptive machine-learning algorithms to give the human IT pro's brain an artificial intelligence (AI) boost -- a concept known as AIOps.
"The world is not going to get easier," said Rick Fitz, senior vice president of IT markets for Splunk, based in San Francisco. "People are already overwhelmed with complexity and data. To get through the next five to 10 years, we have to automate the mundane so people can use their brains more effectively."
Rick Fitzsenior vice president of IT markets, Splunk
Strong enthusiasm for AIOps has spread throughout the industry. Today's analytics products, such as Splunk, use statistics to predict when a machine will fail or the broader impact of a change to an IT environment. However, AIOps systems may move beyond rules-based systems to improve on those rules or gain insights humans won't come up with on their own, said Brad Shimmin, analyst with GlobalData PLC, headquartered in London. Groups of companies will share data the way they share open source software development today and enhance the insights AIOps can create, he predicted.
The implications for AIOps are enormous. Network intrusion detection is just one of the many IT disciplines experts predict will change with AIOps over the next decade. AIOps may be able to detect attack signatures or malicious behavior in users that humans and today's systems cannot detect -- for example, when someone hijacks and maliciously uses an end-user account, even if the end user's identifier and credentials remain the same.
But while AIOps has promise, those who've seen its early experimental implementations are skeptical that AIOps can move beyond the need for human training and supervision.
"AI needs a human being to tell it what matters to the business," LightStep's Sigelman said, based on what he saw while working at Google. "AI is a fashionable term, but where it's most successful is when it's used to sift through a large stream of data with user-defined filtering."
IT pros are always learning, and finding IT workers ready to level up and develop competencies internally quickly became a challenge for organizations adopting DevOps. Find out more about how businesses are fostering DevOps growth on the IT staff in part one.