Switching from Selenium 1.x to WebDriver/Selenium 2 and HtmlUnit

Recently I became aware of the work that is being done to merge the WebDriver and Selenium codebases. The result is Selenium 2, a project that aims to offer the best bits from both.

Having only used Selenium and not WebDriver before, there were a couple of things about WebDriver that piqued my interest. The first is the very clean object-based API that it provides, allowing you to make calls such as:

browser.FindElement(By.Id("q")).SendKeys("Simple Talk");

to type into a text field (in this case the search box on the Google home page). I like the clarity that this syntactical style provides, making it easy to understand existing tests and write new ones.

The second thing was WebDriver’s support for HtmlUnit. HtmlUnit is a Java project that implements a UI-less web browser entirely in memory. This makes the process of browser automation much more efficient, as the overhead of creating a new browser instance is greatly reduced. Unfortunately, since its a Java project, taking advantage of HtmlUnit from .NET hasn’t been a simple prospect up until now (although Steve Sanderson wrote a great blog post about how to do it using IKVM, see here). With the advent of Selenium 2, this process gets a whole lot easier!

The Selenium 2 alpha builds can be found here. I fired up the standalone server in a virtual machine, and create a new WebDriver against that machine as follows:

new RemoteWebDriver(new Uri("http://testVM:4444/wd/hub"), DesiredCapabilities.HtmlUnit());

Note the URL – to create a DefaultSelenium against this instance of the server, the URL would be http://testVM:4444/, whereas to create a WebDriver it is necessary to tack the /wd/hub onto the end.

To compare the performance of using HtmlUnit against Firefox (as I was previously) I copied one of my test fixtures into a new project and migrated the 12 tests over to use a RemoteWebDriver instead of a DefaultSelenium. Since I last wrote about using Selenium Grid, the number of tests has grown from 372 to 2736. To deal with this growth I increased the number of machines in the Grid to 8, the net result of which was that a fixture containing 12 tests running in at most 8 concurrent threads would take about 1 minute 30 seconds to execute. By comparison to this, 12 WebDriver/HtmlUnit-flavour tests running serially (since the Selenium Grid project doesn’t support WebDriver) took only 24 seconds! that’s almost 4x quicker (3.75x, to be exact), despite the lack of parallelism.

This convinced me that it was worth switching all 2736 tests over to use WebDriver, in spite of Selenium Grid’s lack of support for it. So after a couple of hours work migrating my tests and helper methods over to the new API, I was ready to do a full test run. Against the Grid (and its 8 machines), this took almost exactly 3 hours. Using HtmlUnit, the test run completed in 45 minutes for a 4x speedup 🙂

So even in these early stages, the Selenium 2 project is shaping up to be pretty exciting for .NET web testing. I look forward to getting my hands on an early build of Selenium Grid that supports WebDriver, so I can start taking advantage of HtmlUnit in multiple tests concurrently 😀