Thursday, April 25, 2013

Exploratory Testing by James A Whittaker

I heard of Exploratory Testing a year ago in a session at SAP. I got the book and read through it. The various tours concept explained by James makes a good reading.

As i read through the tours, James thought of giving more structure to Free Style Testing was coming through. The fact that various tours are used to structure test plans at Microsoft and Google, makes the concept interesting.

I have compiled the various tours in a one page document. This is to introduce to the reader the concept of various tours. Please do read the one pager to get a feel of the topic. The book is a very easy read. It is more a book that relates to understanding and less technical. A must read for any one, who tests a software.

Tours Description  Examples
Business District GuideBook tour Testing that the functionality matches with that of the User Manual Using F1 help
( Features and Functions the customers use - 20% Code - that gets used 80%) Money tour Testing the 20% of the Code that sells. Demo Systems demos will help to finalise on the test plans for such a tour.
Landmark tour Choose landmarks based on a series of activites and then run through the process.  Landmarks in PO Scenario: PO Number, Material Document, FI Document, Invoice Number etc
Intellectual tour The tour is called Intellectual as it about asking hard questions to a guide. Similarly in the software it is about testing with features that stretch the software to its limits
FedEx tour Package gets picked and delivered at various locations across the world. The items inside the package remains the same. Similarly in a software we have several inputs which we provide, which needs to remain the same in the output. Choosing such data and following them along the lifetime of the data around a cycle.  Item, Quantity, Del Date etc in a PO.
After-hours tour After Hours - Are the time when people are trying to get back home and as the 20% area rests. In case of software it is the time when maintenance tasks , archive data and back up files etc happen.  Checking on the various background jobs that can be created.
Automatic scheduled tasks etc
Garbage collector's tour Those who collect curbside garbage often know neighborhoods better than even residents and police because they go street by street, house by house,and become familiar with every bump in the road.For software, this is like a methodical spot check. We can decide to spot check the interface where we go screen by screen, dialog by dialog (favoring, like the garbage collector, the shortest route), and not stopping to test in detail, but checking the obvious things (perhaps like the Supermodel tour). all menu items, all error messages, all dialog boxes
Historical District Bad Nieghborhood tour Every city worth visiting has bad neighborhoods and areas that a tourist is well advised to avoid. But in case of software bad neighbours are the buggy areas; in case of exploratory testers it is advised to explore such areas. 
( To test Legacy Code) Museum tour Museums that display antiquities are a favorite of tourists. Untouched legacy code is easy to identify by a quick perusal of the
date/time stamps in the code repository or on project binary and assembly files. Such untouched code is tested in the museum tour.
Prior version tour Whenever a product is constructed as an update from a prior version, it is good to run all the scenarios and test cases that applied to the prior version.This will validate that functionality that users are used to is still supported in a useful and usable way in the new product.
Tourist District Collector's tour  Tourists have a habit of collecting and marking of places they have visited. Similarly for a tester, we can try to collect all the possible out put values. 
( 80% of the code that gets used 20%) Lonely businessman tour A lonely businessman does not do much apart from office, work and airport. So one businessman decided to take a hotel farthest and then take different means to reach and ways. The case for an exploratory tester is to decide on the farthest spot in the application and trying to navigate with various inputs and outputs till there.
Supermodel tour This tour is with a sense,"whatever you do, do not go beyond skin deep". Focus is not on funcitionality or real interaction. For a tester take the tour and watch theUI elements.  Rendering, Performance, GUI Refresh, Consisstency of the elements.
TOGOF  tour It is called the "Test One Get One Free" tour. Start the tour by
running your application, then starting another copy, and then another.Now put them through their paces by using features that cause each application to do something in memory and something on the disk. Try using all the different copies to open the same file or have them all transmit data over the network simultaneously. Perhaps they will stumble over each other in some way or do something incorrect when they all try to read from and write to the same file.
Scottish pub tour A tourist once went on a pub tour with a Scottish group and went on a tour. He found places that, he would never have. These places were crowded and difficult to find without experts. Similarly for a tester, he can use external bloggers/user groups to tour the depths of the application
Entertainment District Supporting actot tour The Supporting Actor tour encourages testers to focus on those
features that share the screen with the features we expect most users to exercise.
Certain links that come with the main document. That are always present; but not checked as a main feature. 
( Supportive Features) Bad alley tour This is called the "behind the scenes" tour. In exploratory testing terms, these are the least likely features to be used and
the ones that are the least attractive to users. The bottom most features in the list of features. 
All nighter tour Known as the clubbing tour. The key teerm here is "All Night". For Exploratory testers it is to check whether the application can last? How long can it run before it collapses. Can be similar to a stress test for tangible materials. 
Hotel District Rained out tour The "Rained Out Tour" is a metaphor used to describe a tour; where in the tourist is locked to a room and feels like aborting the tour. In case of exploratory testing, the name is derived to test the "Aborting" of various operations once started. 
(Software is busy when at Rest ) Couch potato tour A couch potato in a group is the one who is lazy and does not participate in any of the events. Similary in an application it is to give the minimal inputs and making the application run the IF-ELSE statements to give an output.  Run reports with minimal input. 
Seedy district Saboteur An attempt to undermine an application in every possible opportunity. 1. Force the software to take some action. 2. Understand the resources required to successfully complete the action and remove/restrict the resources in varying degrees. 
( Vulnerability points in a software) Antisocial tour The Antisocial tour requires entering either the least likely inputs and/or known bad inputs. If a real user would do a, then a tester on the Antisocial tour should never do a and instead find a much less meaningful input.
Obsessive compulsive tour OCD testers will enter the same input over and over. They will perform the same action over and over. They will repeat, redo, copy, paste, borrow, and then do all that some more. Mostly, the name of the game is repetition. Order an item on a shopping site and then order it again to check if a multiple purchase discount applies. Enter some data on a screen, then return immediately to enter it again. These are actions developers often don’t program error cases for.