How do the most effective IT organizations manage and optimize their application infrastructures -- the set of technology components that sits just above the operating system including databases, messaging, applications and Web servers? Over time, a common set of policies, procedures, and processes -- otherwise know as best practices -- has evolved.
Define a change, configuration management process
One of the simplest ways to reduce downtime is to stop introducing changes altogether in an infrastructure. However, for most organizations this isn't realistic because change is an integral and necessary part of application infrastructure configuration management. Further, infrastructure teams are better off anticipating change and putting solid processes in place to manage it.
An effective change process includes:
- Change policy -- the procedures, processes and rules users must follow to implement a change
- User education -- how to follow the process and the consequences of not following it
- A change review board -- those who must review and approve requests for changes
- Rollback facilities -- which types of changes require rollback plans
- Centralized change -- limiting when and how changes occur
Centralize application infrastructure configuration management
All configuration items should be stored in a centrally accessible database -- a configuration management database (CMDB) -- that users, reviewers and managers can access. This best practice includes more than simply collecting configuration items. It involves actively managing infrastructure from this central database and making changes, reviewing them in context, and implementing changes by pushing the contents out to the live infrastructure.
A CMDB provides:
- A central store of configuration items to facilitate modeling, backup and change management
- Centralized and automatic audit trails of changes - making it possible to track them
- Automatic rollback facilities
- Complete version histories of changes
- Fine-grained permissions to configuration items
Sadly, many infrastructure teams don't fully control their environments. While they are the first to get trouble calls, the teams can't mandate that infrastructure changes only go through approved channels, or are approved by them. Tracking these changes offers an early warning on potential issues. Additionally, it provides version histories, which are vital to problem isolation and improve mean time to repair, as well as supply important metrics for focus area of improvement programs.
Reliable metrics and "gold masters"
Key metrics for application infrastructure configuration management are those measures of an effective IT organization, as discussed earlier. Metrics collection, reporting and analysis are other areas to consider. Without automated collection of these metrics, most organizations simply don't have the time to retroactively collect and analyze them.
A "gold master" is a configuration that has been tested and known to work. Many IT teams handcraft the application infrastructure for each new application and environment. Highly effective teams instead reuse gold master configurations. In order to capture, catalog and reuse gold masters, you must have an effective database of these configurations, which is one of several reasons for implementing the best practice of centralizing application infrastructure configuration management.
Automate repetitive and redundant tasks
Even after fully implementing the best practices described above, IT teams spend a lot of time in repetitive tasks that are likely to be error-prone. This best practice suggests that you analyze the activities of your IT team and automate those tasks that they perform repeatedly. Almost any task can be automated, but the tradeoff is in the effort to implement and support the automation. Unless you have a platform on which to implement automations, usually built on a centralized database, the support burden can quickly outweigh the advantages of the automation in the first place. This is the reason many IT teams don't automate in more circumstances.
Overall, wherever your IT organization lands on a spectrum from chaos to a highly-effective, accurate and responsive team, best practices can guide you in improvement. No doubt you already employ some of the practices mentioned and others aren't appropriate to your environment. As with any process guidance, think of these as recommendations, not requirements. Apply the ones that make sense for your organization as they are sure to lead to more efficient operations with higher quality and throughput.
About the Author Bill Thornburg is vice president of product management for mValent. He works with IT organizations to implement effective infrastructure change management with mValent's products.