ChengVoon Tong

7 April 2015

Have you tried team exploratory testing?

If not, why don’t give it a go? As a company which adopts an agile methodology, testing is an integral part of the development process at Redgate. Team exploratory testing is an approach we take so that everyone on a team, from the developers to the support engineers as well as the project and product managers, can get involved in testing the product. I have personally both organised and participated in several team exploratory testing sessions, so I wanted to share some insights about how to run them..

How we run team exploratory testing

Each session has a charter which defines the goal and focus of the test session, and is usually timeboxed to between 60 and 90 minutes. Team members are paired up, with each pair usually comprised of people with complementary skillset, for example a tester and a developer. The main advantage of this arrangement is that each person in the pair offers different insights and can learn from each other – for example, the tester gains insight about the software architecture, and the developer becomes more conscious about risks they might not be aware of.

A tester normally leads and organises a session and, in my experience, preparation is key to making a team exploratory testing session a success. The organiser needs to prepare the instructions for the session, clearly identifying the charter, areas and scenarios, and variations in environments to be tested. Any preparation required needs to be communicated to the participants prior to the session, for example setting up virtual machines and data configuration.

During the session, the setup is normally that one person “drives” the testing, and the other takes notes. Note-taking could include anything from issues, areas explored, reproduction steps and test ideas. We normally write any issues found, such as bugs, surprises, risks and questions, on post-it notes. At the end of the session, the team gathers around the whiteboard and each pair presents what they have explored and their findings. Each of the post-it notes will be put on the whiteboard and categorised using affinity mapping. The team, together with the product manager, will then decide the priority of each issue and their corresponding actions. For things that are not going to be acted upon immediately, the organiser will usually make sure that they are reported in a bug tracking system or filed in relevant backlogs.

From the sessions I’ve run, the most appealing feature of team exploratory testing is the freedom it gives to the participants in deciding what techniques they apply while bouncing ideas off each other. It’s fascinating to observe that this unique interaction tends to lead to the discovery of some interesting bugs that would otherwise not be found. However, participants are not on their own if they cannot come up with ideas – I normally hand out a product map and Elisabeth Hendrickson’s Test Heuristics Cheat Sheet as aid to help generate testing ideas, just in case they need them.

Tips for first timers

If the idea of team exploratory testing appeals to you, here are a few tips on how to make your session a success:

1. Be realistic about what can be achieved in the time available. Don’t ask your team to test a massive area of functionality, which would require hours of set up. If an area to be tested is unfamiliar to most people on the team, it is best to start with a recon session as a way of learning about the area
2. Craft your charter effectively. Ideally the charter targets a specific area to explore and should not be too specific or too broad. In her book Explore It! Reduce Risk and Increase Confidence with Exploratory Testing, Elisabeth Hendrickson mentions that a good charter offers directions without specifying test action
3. Practice before your first run. Before running this with your entire team, organise a trial run session with other testers in your company and get their feedback
4. Offer help to participants. As the organiser, try not to be part of a pair, and instead walk around to offer clarification and help to the participants. For example, you could explain additional risks that participants should consider for test cases they come up with.
5. Be prepared. Allow sufficient time for identifying the charter, the pairings, the machines etc. Make sure you allocate sufficient time for all the work involved in affinity mapping the issues and items found during the session and then either fixing or writing them up afterwards!

If there’s anything else you’d like to know, or any questions you have based on what I’ve written here, let me know in the comments. Good luck with your sessions!