Jose Neto

14 September 2015

Using PhantomJS to improve UI automated testing

Test automation is a major part of the testing effort conducted at Redgate, and all our teams work hard to continuously improve existing practices. In this post, I’ll describe one of the problems we faced – and our approach for solving it.

Scenario: A set of UI automated tests with Selenium were not trustable. Several issues existed, such as errors due to browser initialization on the test VMs causing the build to fail and therefore giving false negatives.

Team’s action: Considering the time we’d already spent trying to make the UI testing structure more reliable, we decided to move from a cross browser testing approach that seemed excessive to a headless browser approach.

What’s a headless browser?

Summarizing it to a short sentence, it’s just another browser but without a user interface.

From a testing perspective, it allows a faster execution time, whilst keeping the level of confidence high. That’s because it uses the WebKit engine, the same one used for Google Chrome, which means that this is not browser emulation but a real engine.

If you’re already familiar with Selenium, moving to PhantomJS is as easy as initializing any of the existing WebDriver implementations. All you need to do is have the service available in your project and create your instance of PhantomJSDriver.

Benefits:

• Reduced maintainability effort, since we stopped worrying about different versions of IE, Chrome, Firefox, and Safari, and focused on making tests stable for the PhantomJS headless browser.
• An easier and quicker way to run tests locally, with faster feedback to the development team enabling them to execute tests more frequently.
• A trustable set of UI tests, where failures indicate real issues, rather than wasting time on fixing structural problems.
• Reduced overall test execution time, with tests running on PhantomJS significantly faster than the same subset of tests on Firefox.

Downsides:

• The risk taken by the team in no longer doing cross browser testing needs to be mitigated by exploratory testing.
• PhantomJS doesn’t support all web standards, so you should refer to its supported list to make sure it fits your needs.

Want to give it a try?

If you’d like to learn more about automated testing using a headless browser, the starting point is the PhantomJS website. There, you’ll find all the information you need to start, together with a bunch of links that can help you figure out how it best suits your needs.

Also if you want to find out more about Selenium, there are lots of resources on the web, and I recommend you start by visiting their website and watching the available conference videos.