Essential Guide

A look inside the DevOps movement

A comprehensive collection of articles, videos and more, hand-picked by our editors

Agile exploratory software testing: Why IT inefficiency pays off

Agile exploratory software testing is inefficient by traditional IT standards. Which is why it belongs in modern, end-user-centric IT strategies.

FROM THE ESSENTIAL GUIDE:

A look inside the DevOps movement

+ Show More

Giving development testers and operational bug-fixers extra time to break software can improve IT efficiency in an Agile organization.

Agile developers and DevOps specialists use canned, pre-defined test scripts; in fact, Agile devotees see this as part of Agile development in such techniques as Scrum. However, exploratory testing is a key break with past development truisms, and it is part and parcel of Agile IT's reasons for success.

Agile best practices don't mesh with traditional views of IT efficiency. They emphasize allotting time for learning, keeping developers on the project rather than plugging their spare time into other projects and deferring some tests until later in the project.

Traditional software testing is close-ended, using rote methods to test all parts of a piece of code. These methods, practically speaking, never cover all the operational circumstances that an application faces -- especially when that application is constantly upgraded. By contrast, exploratory software testing goes outside the box, discovering unusual circumstances and ways in which the rote tests are inadequate. Exploratory testers aim to find something wrong with the code, and so they find it.

Some exploratory testing adherents without an overall Agile IT approach find that overemphasizing the technique will waste time and money. Not so. Because Agile development involves rapid iterations of a "Here's what I want. OK, how about this as well?" dance with the customer, pre-cooked scripts simply cannot handle all the new cases that come up in the process of innovating features. Therefore, exploratory testing works hand-in-glove with available scripts and test harnesses to test each iteration with Agile development.

Exploratory testing involves the admin fully and continuously in the development process. Testers don't just react to massive chunks of developer code; they act proactively to think of ways to break it.

The most daring extension of the Agile exploratory testing approach is to the Agile side of operational app testing: Agile exploratory DevOps. The administrator fights fires and actively thinks of ways to break the code before problems happen. It requires "operational sandboxes" in which to test.

Top line, bottom line and risks

The benefits of Agile exploratory testing cannot be fully separated from those of Agile development, bug-fix and an upgrade itself:

  • Agile does not focus on costs, but it cuts costs by 20% to 30% compared to a cost-focused development approach.
  • Agile does not focus on revenue or return on investment, but it increases ROI by 20% to 30% compared to a profit-focused approach.
  • Agile does not focus on risk, but it improves quality by 20% to 30% compared to a quality-focused approach.

Agile continually redefines the project goal based on customer feedback, rather than pre-defining what the customer wants and then narrowing the list of features to achieve the release.

Agile exploratory software testing enables and supports the Agile development process, wherein developer code passes the exploratory early-warning tests quickly, and only those parts of the code that are likely "keepers" go through the full-scripted integration-testing process.

Agile exploratory testing fits particularly well with an Agile development practice called "middle-in" development, in which the developer starts with a low-level software infrastructure and puts templates for app functions on top of it, then creates additional infrastructure on top of the basic infrastructure to provide functionality for the template applets. Continual iterations build the features and common infrastructure as needed. Agile exploratory tests are deferred until developments are well defined, and ad hoc tests are repeated in each iteration as evolved test scripts.

Putting the emphasis on innovation

Agile exploratory testing fits better with business needs than simply treating testers like highly controlled, highly optimized test harnesses. Businesses that emphasize innovation do better than those that emphasize costs, according to Sloan Management Review, and that carries through to the IT department.

Agile exploratory testing asks that testers be innovative, and testers testify that they add value to the code beyond simply fixing bugs. The result is a more innovative application.

Agile exploratory software testing makes Agile development and DevOps work better, faster and cheaper, but there is a culture shock. We see Agile developers being inefficient and must learn to accept inefficiency to get the true payoff.

As long as testing doesn't top your priority list, ease into Agile exploratory testing. Start with a "sandbox" for application lifecycle management (ALM) with Agile exploratory testing, then integrate with Agile development and introduce exploratory testing tools and training in a DevOps framework. The time that Agile exploratory testing takes away from fighting daily IT fires is paid off in increased quality and customer satisfaction.

About the author:

Wayne Kernochan is president of Infostructure Associates, an affiliate of Valley View Ventures, with more than 20 years of experience in IT industry analysis. This document is the result of Infostructure Associates-sponsored research. Infostructure Associates believes that its findings are objective and represent the best analysis available at the time of publication.
WKernochan@aol.com

This was last published in September 2013

PRO+

Content

Find more PRO+ content and other member only offers, here.

Essential Guide

A look inside the DevOps movement

Join the conversation

4 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

I would not confuse checks for 'acceptance criteria' as predefined tests.  There is far more to testing than just simply conformance to agile requirements.  I also question the ID that Traditional IT 'efficiency' was ever something that was achievable.   (and it was not always close ended, despite what this article says.)

Plus all testing is exploratory to some degree.   The issue is that rote and rigorous sticking to 'scripts' in the Factory style model of Software testing, is only going to find issues when teams don't produce predictible software.   Otherwise they aren't likely to find issues, because they stick to a simplistic view of application use.


Cancel
The issue I see with this is one that Veretax has already outlined. All testing is exploratory initially, and the canning of repeatable steps through automation can provide a level of checking and repeatability for many aspects of testing, but it cannot replace usability or human factors testing, nor can it do anything you haven't thought of yet. The thought process behind the tests performed needs to be continuously enhanced and updated to allow for searching for things thus far not considered.
Cancel
I think that Michael and Veretax both make good points, and we cannot replace usability or human factors testing. With respect to the next part of Michael’s answer, I think automation can do things that we haven’t explicitly thought of if we implement a model-based approach to automation. In that case, automation can take any path that is allowed in the model, even if we have not thought of taking that path ourselves. Thus, a model-based approach allows the automation script to seek out those things that we have not yet considered.
Cancel
Agile exploratory testing absolutely does fit better with business needs. We are lucky to have dev managers who defend our exploratory testing practices. It may seem inefficient to some, but here is never a doubt in my mind that we are adding a ton of value with our use of time.
Cancel

-ADS BY GOOGLE

SearchDataCenter

SearchAWS

SearchSOA

SearchServerVirtualization

SearchCloudApplications

SearchCloudComputing

Close