One of the biggest oversights in any data center is the lack of clear and informative server documentation.
IT professionals know how to install and configure the most complex equipment, but often don't have strong communication skills. As a result, organizations rely on administrators' memory or informally passed-along knowledge, causing avoidable integration, upgrade and troubleshooting problems because administrators forget what they've done or leave behind a knowledge vacuum.
Here are some strategies for improving your server documentation.
Strong server documentation
There is no single established standard for system or server documentation. The emphasis is not on the path, but rather on establishing a "map" of how each system is equipped, configured and integrated into the data center so that other IT professionals can understand, test, upgrade and troubleshoot it and the production environment with minimal time wasted.
Organizations document their servers and systems differently, using vastly different criteria and levels of detail. But there are ways to shore up haphazard documentation.
Lists. Start with a hardware inventory that includes a list of all major components. Follow that up with a software inventory that details the operating systems, hypervisors, virtual machines (and all their operating systems), drivers, applications (workloads), and all the associated licensing information.
Any system inventory tool can detail hardware and software components and update over time, but a printed component list offers a complete at-a-glance picture, which can easily be compared to systems' purchase specifications. Organize inventory reports along with original installation or recovery media and any original vendor documentation, such as manuals or setup guides.
Instructions. Next, document anything related to each individual system's setup and configuration, starting with its firmware options (BIOS settings). Also, document startup scripts. Command-line scripts notoriously lack comments, so add them for easier changes or troubleshooting in the future. Then, implement a version control system to help administrators track the version of each one and understand when a system might be using old or erroneous scripts.
Document the system's integration into the larger data center, including its LAN address, the media access control, or MAC, address for each network interface card's port, and external notes that show how the system is interconnected with network switches. Taken together, this information forms a map of the network for IT professionals to follow to survey current architectures and make suggestions for infrastructure improvements.
Systems management tools automate at least some of these information-gathering tasks. Tools frequently replace more formalized documentation because reporting changes dynamically and won't become obsolete. No matter the method, implement a process to update systems and their associated documentation as changes occur. Wrong documentation can be worse than no documentation.
Where is that document?
As is the case with system and server documentation content, there is no clear choice for where to keep it.
System documentation should ideally be located close to the physical system, speeding access to important details when uptime is on the line. Keep documentation on something other than the system it represents. Documentation is meant to help with troubleshooting, but saving notes on the system's local hard drive won't do much good if that system fails or becomes inaccessible.
Consolidating documents in a single location is also common. If you choose this route, set aside a shelf for manuals, configuration files and other details in the data center. Avoid redundant copies of documentation -- changes to one copy often don't get migrated to subsequent copies, resulting in errors and confusion.
The document medium itself makes little practical difference. Paper printouts, files on optical discs, records on flash drives -- even a library of electronic manuals and documentation files on the primary storage area network: These are all acceptable media depending on the size and complexity of your environment. Systems management tools might generate very few hard-copy documents. Ensure consistency and consolidation so that each set of details is readily available.
Handle server documentation with care
Generally, the vendor documentation that accompanies a new server has limited value once the system is actually configured and deployed, but it is good practice to retain vendor documentation with new documentation you develop.
Vendor documentation bulk and clutter is becoming less of an issue as more vendors shift to Web-based documentation and guides. For example, vendors can update their online documentation alongside an evolving knowledge base to help deal with errors, oversights, troubleshooting and workarounds.
Still, the original documentation and current configuration details add value when an aging system is repurposed (sold to another user or reassigned to another business unit).
For your documentation convenience
Documentation must be clear and precise -- down to the specific keystrokes or mouse clicks of a complex setup and configuration procedure. It is possible to document detailed activities manually, but the potential to skip over certain steps is high.
Tools record processes and give documentation users a step-by-step insight into key procedures.
One tool is Microsoft's Problem Steps Recorder (PSR.exe), originally introduced with Windows 7 and Windows Server 2008 R2 to help support staff see what's happening on remote users' desktops. IT administrators recognized that the ability to record and comment on each mouse click and then save that activity into a zipped MHTML report could be used for more than remote user activities.
The tool, continued in Windows Server 2012, can record and document important activities for client- and server-side systems. It creates a library of recordings that staff can refer to when they tackle important setup and configuration activities. To launch PSR.exe, click Start, type psr into the search line, then click on the PSR applet that appears in the search results list.
There are no established standards for documenting any part of a data center, so documentation quantity and quality vary from one organization to another. Peer-review improves documentation, allowing other IT staff to read material and provide feedback on its clarity and completeness. And data center managers should make time for periodic training, allowing IT staff to familiarize themselves with the available documentation before they need it.