ChengVoon Tong

12 January 2015

1 Comment

Top 3 tips for software testing a mature product

Developers and testers generally don’t like to work on a mature product with a legacy code base, especially when the majority of the team has no previous knowledge of either. Having said that, while building and testing a mature product can prove a lot more challenging than tackling a greenfield one, my recent experiences suggest there are things that testers can do to minimise the pain.

Map Out the Product

Early on in the project you should spend some time exploring the software, and trying to model the features and requirements of the product. A graphical model (for example, a mind map) can provide a concise, easy-to-understand representation of the product, and the modelling process is likely to help you uncover features that you may not previously have been aware of.

Initial version of our product map

However, a word of warning – you may find it challenging or even frustrating to break down the application in this way, especially when certain sub-features can appear in multiple different top-level features, or when certain parts of the application don’t fit nicely into any particular category. Don’t let this bother you too much – it should be fine as long as you and your team can agree on a taxonomy and easily understand the model. Apart from exploring the application, you can also learn about the product by talking to your product manager, watching a sales demo or referring to the product’s documentation (if it exists).

Visualise Your Progress and Coverage

You can use your product map to help you model the existing test suites and test environment. Having a handy list of types of testing can guide you in identifying what the existing test suites do and don’t cover:

• Are there any tests that you can automate?
• Can you see any existing tests that don’t serve much purpose?

Code coverage tools can be your friends, but you need to interpret the results with care! Low coverage figures highlight areas in the code base that lack tests, but a 100% coverage doesn’t necessarily mean you have good test coverage in a particular area.

It’s very important not to let your product model become a static document if you really want to benefit from it. My team inherited a massive scripted release testing checklist which would involve hundreds of test cases and 500 man hours for a major release, but we decided to cut down the testing and be more engaged and context-driven in our release testing. We used mind maps to aid our fortnightly minor release testing, with everyone on the team (not just the testers) spending an hour testing designated areas. Over time, we refined the mind maps by consolidating features that are functionally similar and expanding on the more ‘risky’ areas we have identified.

More recently, for our major release testing, we turned our mind maps into heat maps to help us visualise how our testing progressed, and highlight areas that needed more attention. As a result of being more targeted, we not only more-than-halved the time we spent on the release testing, but also found a larger number of relevant issues, and fixed them sooner.

Heat maps to help us visualise our testing progress

Acquire Domain Knowledge

If your product requires very specific domain knowledge and involves complicated workflows, you will greatly benefit from acquiring at least some of that relevant knowledge. I have no doubt that any tester can easily find bugs without deep knowledge about the software under test, but if you want to find issues that really matter, then I think you really need to invest some time to properly understand the product (including user scenarios and the underlying technologies that the product relies on). This is likely to be an evolving process, though – I have been amazed by the fact that I learn something new about Microsoft SQL Server (which is the underlying technology of many Redgate Software tools) every time I test a new feature or retest a bug in our product.

“We debate and agree on the acceptance criteria and testing requirements of a feature up front.”

Equipped with good domain knowledge, testers can really make an impact by performing proactive testing. When we started working on our recent mature product, my team got frustrated with last minute requirement changes and surfacing of issues, so we decided to adopt the “Three Amigos” for our features planning, where each session involves a product manager, a developer and a tester. With this approach, we debate and agree on the acceptance criteria and testing requirements of a feature up front, and the developer and the tester must be people who will be involved in the actual development work. I’ve seen my team greatly benefited from adopting such a collaborative approach, and having the domain knowledge empowers you in these discussions, and helps highlight risks early on.

Finally…

It’s natural for you to not feel excited when learning that you’ll be working on a legacy product, and some people may even be pessimistic. However, don’t let that negative feeling follow you throughout the project! I believe that such experience can make better use of your testing skills and shape you into a better tester, and these skills can (and should) be applied to every project you work on.