This content is part of the Essential Guide: A look inside the DevOps movement

Agile exploratory software testing: Why IT inefficiency pays off

Agile exploratory software testing is inefficient by traditional IT standards. Which is why it belongs in modern, end-user-centric IT strategies.

Giving development testers and operational bug-fixers extra time to break software can improve IT efficiency in an Agile organization.

Agile developers and DevOps specialists use canned, pre-defined test scripts; in fact, Agile devotees see this as part of Agile development in such techniques as Scrum. However, exploratory testing is a key break with past development truisms, and it is part and parcel of Agile IT's reasons for success.

Agile best practices don't mesh with traditional views of IT efficiency. They emphasize allotting time for learning, keeping developers on the project rather than plugging their spare time into other projects and deferring some tests until later in the project.

Traditional software testing is close-ended, using rote methods to test all parts of a piece of code. These methods, practically speaking, never cover all the operational circumstances that an application faces -- especially when that application is constantly upgraded. By contrast, exploratory software testing goes outside the box, discovering unusual circumstances and ways in which the rote tests are inadequate. Exploratory testers aim to find something wrong with the code, and so they find it.

Some exploratory testing adherents without an overall Agile IT approach find that overemphasizing the technique will waste time and money. Not so. Because Agile development involves rapid iterations of a "Here's what I want. OK, how about this as well?" dance with the customer, pre-cooked scripts simply cannot handle all the new cases that come up in the process of innovating features. Therefore, exploratory testing works hand-in-glove with available scripts and test harnesses to test each iteration with Agile development.

Exploratory testing involves the admin fully and continuously in the development process. Testers don't just react to massive chunks of developer code; they act proactively to think of ways to break it.

The most daring extension of the Agile exploratory testing approach is to the Agile side of operational app testing: Agile exploratory DevOps. The administrator fights fires and actively thinks of ways to break the code before problems happen. It requires "operational sandboxes" in which to test.

Top line, bottom line and risks

The benefits of Agile exploratory testing cannot be fully separated from those of Agile development, bug-fix and an upgrade itself:

  • Agile does not focus on costs, but it cuts costs by 20% to 30% compared to a cost-focused development approach.
  • Agile does not focus on revenue or return on investment, but it increases ROI by 20% to 30% compared to a profit-focused approach.
  • Agile does not focus on risk, but it improves quality by 20% to 30% compared to a quality-focused approach.

Agile continually redefines the project goal based on customer feedback, rather than pre-defining what the customer wants and then narrowing the list of features to achieve the release.

Agile exploratory software testing enables and supports the Agile development process, wherein developer code passes the exploratory early-warning tests quickly, and only those parts of the code that are likely "keepers" go through the full-scripted integration-testing process.

Agile exploratory testing fits particularly well with an Agile development practice called "middle-in" development, in which the developer starts with a low-level software infrastructure and puts templates for app functions on top of it, then creates additional infrastructure on top of the basic infrastructure to provide functionality for the template applets. Continual iterations build the features and common infrastructure as needed. Agile exploratory tests are deferred until developments are well defined, and ad hoc tests are repeated in each iteration as evolved test scripts.

Putting the emphasis on innovation

Agile exploratory testing fits better with business needs than simply treating testers like highly controlled, highly optimized test harnesses. Businesses that emphasize innovation do better than those that emphasize costs, according to Sloan Management Review, and that carries through to the IT department.

Agile exploratory testing asks that testers be innovative, and testers testify that they add value to the code beyond simply fixing bugs. The result is a more innovative application.

Agile exploratory software testing makes Agile development and DevOps work better, faster and cheaper, but there is a culture shock. We see Agile developers being inefficient and must learn to accept inefficiency to get the true payoff.

As long as testing doesn't top your priority list, ease into Agile exploratory testing. Start with a "sandbox" for application lifecycle management (ALM) with Agile exploratory testing, then integrate with Agile development and introduce exploratory testing tools and training in a DevOps framework. The time that Agile exploratory testing takes away from fighting daily IT fires is paid off in increased quality and customer satisfaction.

About the author:

Wayne Kernochan is president of Infostructure Associates, an affiliate of Valley View Ventures, with more than 20 years of experience in IT industry analysis. This document is the result of Infostructure Associates-sponsored research. Infostructure Associates believes that its findings are objective and represent the best analysis available at the time of publication.
[email protected]

Dig Deeper on Scripting, Scheduling and IT Orchestration