kran77 - Fotolia
Small World Big Data
Published: 16 Nov 2017
As a technology evangelist and professional IT systems optimizer, I see the benefits of IT automation and can only champion trends that increase it. When we automate onerous tasks and complex manual procedures, we naturally free up time to focus our energies higher in the stack. Better and more prevalent automation increases the relative return on our total effort so that we each become more productive and valuable. Simply put, IT automation provides leverage. So it's all good, right?
Another IT automation benefit is that it captures, encapsulates and applies valuable knowledge to real-world problems. And actually, it's increasingly hard to find IT automation platforms that don't promote embedded machine learning and artificially intelligent algorithms. There is a fear that once our hard-earned knowledge is automated, we'll no longer be necessary.
So, of course, I need to temper my automation enthusiasm. Automation can eliminate low-level jobs, and not everyone can instantly adjust or immediately convert to higher-value work. For example, industrial robots, self-driving cars or a plethora of internet of things (IoT)-enabled devices that cut out interactions with local retailers all tend to remove the bottom layer of the related pyramid of available jobs. In those situations, there will be fewer, more-utilized positions left as one climbs upward in skill sets.
Still, I believe automation, in the long run, can't help but create even more pyramids to climb. We are a creative species after all. Today, we see niches emerging for skilled folks with a combination of internal IT and, for example, service provider, high-performance computing, data science, IoT and DevOps capabilities.
Automation initiatives aren't automatic
A service provider has a profit motive, so the benefit of IT automation is creating economies of scale. Those, in turn, drive competitive margins. But even within enterprise IT, where IT is still booked as a cost center, the drive toward intelligent automation is inevitable. Today, enterprise IT shops, following in the footsteps of the big service providers, are edging toward hybrid cloud-scale operations internally and finding that serious automation isn't a nice-to-have, but a must-have.
If one squints a bit, almost every IT initiative aims to increase automation. Most projects can be sorted roughly into these three areas with different IT automation benefits, from cost savings to higher uptime:
- Assurance. Efforts to automate support and help desk tasks, shorten troubleshooting cycles, shore up security, protect data, reduce outages and recover operations quickly.
- Operations. Necessary automation to stand up self-service catalogs, provision apps and infrastructure across hybrid and multi-cloud architectures to enable large-scale operations, and orchestrate complex system management tasks.
- Optimization. Automation that improves or optimizes performance in complex, distributed environments, and minimizes costs through intelligent brokering, resource recovery and dynamic usage balancing.
Automation enablers at large
Successful automation initiatives don't necessarily start by implementing new technologies like machine learning or big data. Organizational commitment to automation can drive a whole business toward a new, higher level of operational excellence.
For example, Hewlett Packard Enterprise's Nimble, back when it was just Nimble, started with a specific IT automation vision. It would strive to eliminate level 1 and 2 support altogether through diligent and relentless internal automation. This has paid off for both the vendor and its clients with a reported 86% of the issues that would formerly be reactively dealt with by level 1 and 2 support teams being both opened and pre-emptively solved automatically.
Of course, creating an automation culture depends on recruiting smart, motivated level 3 engineers -- who amusingly could also be seen as inherently lazy and self-serving enough to automate their way out of repetitive tasks. Cynics among us might think words are cheap. You don't reap IT automation benefits without doing the hard work of implementation. Luckily, organizations today can take advantage of a happy confluence of enabling trends that include broad and deep rule and policy and orchestration engines, increasingly programmable infrastructure and DevOps-style utilities.
At the infrastructure level, key resources are now available in software-defined form. While we could debate what the overused software-defined term has really come to mean today, in practice, software-defined storage, networking and other services tend to come with extensive APIs that enable remote, dynamic programmability. IT can use those APIs to automate provisioning, configuration and optimization, automatically tailoring resources to current conditions and changing requirements. At the same time, those APIs can often produce useful insight into current utilization, ongoing status and queuing bottlenecks, which can all be used to trigger automated management adjustments and dynamic optimization.
On the application side, the drive toward Agile DevOps has encouraged IT features that enable automation at scale in complex environments. Applications can now be provisioned through blueprint files that describe key resources, components and services that can then be automatically deployed at-will to stand up a full application environment. In fact, the container world is built around these concepts, with editable manifest files that describe how each container's environment should be automatically set up on deployment.
At a lower level, both IT and DevOps application owners can make good use of popular tools such as Chef, Puppet and Ansible, along with their growing and extensive task libraries. Using these scripting utilities, or an increasingly common drag-and-drop automation engine from their favorite vendor, a motivated IT expert can readily automate infrastructure provisioning, application deployment and system management tasks, and better ensure solid configuration management. Systems-level automation deserves best practice configuration and change management control as much as code patching and infrastructure cutover activity do.
Practical AI is automation intelligence
I predict a convergence of IT automation offerings so that we can end up with a single smart brain capable of covering the complete IT stack of responsibility -- from infrastructure operations to service assurance. And complete automation platforms should not only react to events through triggers, but also should proactively enforce all kinds of policies: service level, security, cost control and so on. The bigger and broader the automation scope, the more big data analysis and machine learning functionality it will likely require to conquer the resulting scale and complexity.
I expect that the advanced learning necessary for automating large-scale heterogeneous hybrid cloud operations will require a service provider collection of baseline data from multiple clients. This means the most sophisticated opportunities for automation may be beyond the individual enterprise's internal span of insight. This possibility could account for the increasing success of remote, managed IT service offerings -- what I've been calling management as a service. By subscribing to such an offering, a client takes advantage of the provider being heavily motivated to use its broader, multiclient perspective to create and embed as much intelligent automation as possible.
If all that weren't enough, the oncoming wave of IoT and industrial IoT will absolutely require advanced automation. The future is fast bringing tens of thousands of containers, multiple petabytes of big data and millions of edge IoT computing devices to the enterprise.
The good news for IT folks is that because every day brings something new, IT may never become a fully automated self-driving black box. But there is a big challenge: Humans need to use intelligent automation as much as possible to try to keep up with the ever-increasing scale and complexity.
IT department? What IT department?
BizDevOps jostles IT team into the business's realm
On the hunt for automation gurus and DevOps engineers