Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Nine DevOps metrics you should use to gauge improvement

What's missing from your DevOps pilot and scale-up plans? Cold, hard, impartial numbers. Track these key metrics to ensure DevOps processes achieve stated goals.

It's no small task to transform an IT organization to integrate development, operations and quality assurance teams. A DevOps methodology requires team and process changes and then, once everything is in place, the onus is on IT to create DevOps metrics and measure the outcomes.

The key to a productive DevOps program is effective and comprehensive measurement and monitoring. Create a detailed game plan to understand how processes and projects work and how to improve them over time.

DevOps metrics are complicated by the fact that this methodology has no formal framework, thus the definition of progress can vary between organizations. However, there are common metrics and key performance indicators that should keep a DevOps implementation on track.

Deployment frequency

Innovation begets rapid technology changes. Track code deployment frequency for a clear picture of how quickly new features and capabilities roll out. This metric should remain stable or trend upward over time. A decrease or stutter could indicate a bottleneck somewhere within the DevOps team structure.

Change lead time

Determine how long it takes a change such as a bug fix or new feature to go from inception to production. Long lead times could suggest inefficient processes that inhibit change implementations. This DevOps metric should be a starting point to discover patterns that signal complexity that is causing a bottleneck in a given area.

Change success or failure rate

Changes must roll out smoothly, not just frequently. Keep the failure rate for changes deployed into production applications as low as possible. Develop a system to track the success and failure of changes. A high rate of change failure affects the application's end users, and requires that admins invest additional time to troubleshoot issues and fix bugs rather than accomplish high-value initiatives.

Mean time to detection

Don't measure DevOps metrics in isolation. A low change failure rate isn't good enough if it takes too long to detect a problem. For example, if the MTTD is thirty days, that means it could take a full month to diagnose an issue that causes failure rates to rise. MTTD should decrease over time as DevOps processes mature. If MTTD is high or trends upward, expect the bottlenecks causing these existing delays to introduce additional congestion into the DevOps workflow later down the road. Fast detection is a boon to security as well, as it should minimize an attack's reach.

Mean time to recovery

MTTR is another DevOps metric that admins should keep as low as possible. Eliminate issues once you become aware of them. DevOps organizations follow the principle that frequent, incremental changes are easier to deploy, and easier to fix when something goes wrong. If a release includes a high degree of change and complexity, it becomes more difficult to identify and resolve issues within it.

KPI changes.
Key performance indicators, like the DevOps organizations that use them, change and evolve over time.

Feature prioritization

How often do end users take advantage of the product's or service's features? If your DevOps team spends the time and effort to create and implement new code into production, make sure that those features are valuable to the target population. BizDevOps reinforces the value of iteration to meet user demand as quickly as possible. If feature usage is low or declining, reevaluate the prioritization factors with management to determine a better feedback loop from users.

Rate of security test passes

Vulnerability scanning is not a common test case within a release pipeline. Determine the test's viability and implement whenever possible. A failed security test during the build phase could prevent some releases from going into production. If code releases consistently fail vulnerability tests, this DevOps metric suggests that teams are ignoring secure practices upstream and ultimately must correct those issues.

Service availability

Application uptime is an important metric for every IT organization. Service-level agreements require that the infrastructure, services and supporting applications meet a high goal of availability. Services should be online as much as possible, which creates uptime goals as high as 99.999%.

Application performance

An unexpected influx of end users can create performance issues at the infrastructure layer -- turns out you can get too much of a good thing. Storage bottlenecks, CPU spikes, high memory consumption and network latency are all common side effects of a surge in application use. Closely monitor these standard performance aspects of the servers that support an application. Increasing volumes of end users can require additional infrastructure to be built in. However, performance decreases without additional end-user requests could indicate that bugs or inefficient changes from development and release are bogging down the app. Verify and correct these as they occur to enable high availability and a good end-user experience.

Next Steps

Make BizDevOps happen in your org

How mature are your IT ops processes?

Follow this guide for an IT overhaul

This was last published in October 2017

Dig Deeper on DevOps and IT Certifications and Training

Join the conversation


Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

What are your organization's primary DevOps metrics and KPIs?