Hello, help desk, how can I help you?
I have a problem with this app -- I could do this yesterday; today, I can't.
Hang on -- I can't see that on my screen. Ah, it's those developers again. They've pushed out some new functionality, and we aren't aware of it. Sorry, but I can't help; thank you for your call!
These are the battles of DevOps support -- in particular outsourced support. In situations like these, uncontrolled DevOps adoption idolizes change and speed without consideration of the help desk and users who rely on them.
The promise of DevOps is that it gets much-needed functionality to the business rapidly, using the various C's of CI/CD: continuous integration, continuous development, continuous delivery and continuous deployment. IT organizations generally believe that users can absorb small, incremental changes introduced through a DevOps methodology without swamping the help desk with incidents and requests. What happens when the change is larger or has just been implemented badly?
In-house teams often discover DevOps support challenges, such as communicating changes to users at an increased rate without overloading them. But at least they have direct access to the company's DevOps teams, and those people act as second- or third-level support. When the support, at any level, comes from a third party, it makes things harder.
The wrong metrics in support
The metrics in support contracts generally are based around time to fix or time to close of issue. These metrics work with reasonably long periods of fixed functionality, as provided with cascade project management, but falter when changes are introduced on a regular basis.
To decide on new metrics that validate an outsourced DevOps support agreement, start with a check of existing metrics -- and probably kill a lot of them off.
Some help desk key performance indicators (KPIs) are easy to game. KPI for number of calls answered? Just pick the call up, and then drop it again -- the requestor is left no better off and angry, but the KPI is met. Simple time to resolution is similar: Too many calls are marked as resolved by the help desk agent, rather than by the user. The user then has to raise the same issue again as a new call if it was not truly fixed.
Once spurious KPIs such as these are dealt with, then focus on what a DevOps support environment really needs.
Good DevOps support gets everyone involved
Support must be firmly connected to the whole DevOps process -- the help desk is not something to be tacked on as an extra at the end. Keep the support team aware of what is coming down the line, when it is going live, who it is going live to and the business importance of each change.
Give support staff early access to new functionality during the test phase, before changes go live. A true DevOps process enables the IT team to put code live to a small group: Part of the support team can try out the new functionality in line with the rest of the live system to find potential issues and gotchas. The early access group can rotate so that everyone in support -- or at least a significant portion of the team -- sees the changes.
Use skills routing to make DevOps support work if the entire help desk cannot preview changes. Skills routing ensures that user requests for help go through to a worker who is up to speed on changes. It's no good to involve help desk in early access if a call on the change gets sent to an agent who is not up to speed.
Feedback loops are integral to the DevOps process. A user call alerts the IT organization that code needs to change; developers need this information flagged to them as soon as possible. Don't rely on older-style help desk ticketing systems. Instead, invest in modern DevOps control systems with change management capabilities.
Where possible, assign issues a business priority level. Users can provide a score for how much the issue affects their capability to get on with work. Where the DevOps support team identifies clusters of the same issue, it should force the issue up the priority list.
How are we doing?
A DevOps support setup needs metrics, too, but not the same ones you're used to: It starts with proper resolution measurement then tracks user satisfaction.
One-size-fits-all metrics are meaningless. For example, don't force the DevOps support team to resolve all issues within one working day. A password reset, for example, is generally too important for a user to wait a whole working day. Conversely, an issue where the root cause points to a complex problem should not be rushed just to fit one spurious metric.
Ensure that expectations are always set and then managed. For example, tell users that the help desk can reset a password while on the call. Or for more complex issues, inform the user that they will get an update on the case within the hour with a better idea of how long its resolution will take.
The user should mark a case as resolved, not the support technician. Ensure the loop closes by sending them simple one-click reminders. At the same time, survey users on how they felt the issue was handled then analyze the results to improve DevOps support.
Jayne Groll of the DevOps Institute calls for Agile in IT service management approaches.
User feedback could help identify a bottleneck between the help desk and the developers or the operations staff, which should be treated with urgency. Lessons learned from user surveys also emphasize the IT-business relationship. Developers tend to reprioritize issues according to technical interest, rather than business value -- let the user input remind them who pays their salaries and where the money comes from.
This approach to support is nuanced and complex -- it will be difficult to enforce such a contract between in-house and outsourced teams. Offer rewards more than punishments. An outsourced team that's penalized when things go wrong will point fingers rather than work harder. Financially reward both the outsourcing company and the internal staff when problems are dealt with in a timely, efficient and effective manner. All sides see a positive stimulus as an incentive to work together.
DevOps support does require some changes, but these should not be onerous. Nurture joined-up thinking, a high degree of communication and collaboration across the whole process. Without it, a DevOps strategy is doomed to fail.
DevOps can go wrong even with the best intentions. Sometimes, it's the result of old habits carried over into a new way of working. Other problems come down to IT and business leadership or training. Common DevOps myths also hold back teams and companies from success.