EDIT: After a request in the comments, the full solution can be obtained from here.

After my last post it was suggested to me that I create a [URL] attribute to deal with the unnecessary repetition of the testStartPageUrl parameter when creating a [Browser] attribute. The example below shows the problem:

[Theory]
[Browser(Browsers.InternetExplorer7, "http://www.google.com")]
[Browser(Browsers.Firefox3_5, "http://www.google.com")]
[Browser(Browsers.GoogleChrome, "http://www.google.com")]
[Browser(Browsers.Opera, "http://www.google.com")]
public void Google_For_SimpleTalk(ISelenium iSelenium)
{
    //Test code
}


In each case it was necessary to specify “http://www.google.com”, to be passed through to DefaultSelenium’s constructor. However, it would be much cleaner if we were able to write something like the following:

[Theory]
[URL("http://www.google.com")]
[Browser(Browsers.InternetExplorer7)]
[Browser(Browsers.Firefox3_5)]
[Browser(Browsers.GoogleChrome)]
[Browser(Browsers.Opera)]
public void Google_For_SimpleTalk(ISelenium iSelenium)
{
    //Test code
}


Getting this much running was straightforward enough, thanks to a tip I received from Matt Ellis. He suggested that I take advantage of MethodInfo.GetCustomAttributes() from within the [Browser] attribute, to grab the [URL]. So I did just that! I added a private method to the BrowserAttribute responsible for populating its private _url, and then called this in my override to GetData():

private void GetURLFromAttribute(MethodInfo methodInfo)
{
    var urlAttribute = (URLAttribute) methodInfo.GetCustomAttributes(typeof (URLAttribute), false).FirstOrDefault();
    _url = urlAttribute.Url;
}

public override IEnumerable<object[]> GetData(MethodInfo methodUnderTest, Type[] parameterTypes)
{
    GetURLFromAttribute(methodUnderTest);
    yield return new[] { new[] { CreateBrowser() } };
}


You may have noticed that the above code has a problem. Specifically, if I were to omit the [URL] attribute my code would pass null to the DefaultSelenium constructor. This of course causes an exception to be thrown when I attempt to call Start() on the Browser object. I obviously needed a null check, so I inserted one at the beginning of the CreateBrowser() method:

if (_url == null)
throw new NullReferenceException("No [URL] attribute was defined");

At this point I was unfortunate enough to run into a bug in xUnit 1.1 🙁 Any exception thrown by a derived DataAttribute was not being handled by the test runner, causing it never to proceed beyond that failed attempt to retrieve test data. Fortunately for me, xUnit 1.5 shipped at the weekend, which included the fix for this issue 🙂

With that problem sorted, however, I immediately encountered another one. Moving from xUnit 1.1 to 1.5 seemed to have re-introduced the “start all the browsers before any test runs are executed” behaviour I eliminated last time out. On looking at the changes made in 1.5, it turns out that the same  change that fixed the exception issue was causing the browser start up problem! Briefly, the code that enumerates test data for DataAttributes went from yield returning new TheoryCommand()’s for each attribute, to adding them to a collection and returning that at the end of the loop.

It was clear that to once again fix this issue, I was going to have to delay calling the browser object’s Start() method. Previously, I had been doing this within the BrowserAttribute.CreateBrowser() method, so when the enumeration code called my override to GetData() it was being passed back a started browser object.

To achieve the delayed start, I decided to wrap the browser in object in a new SeleniumProvider object, which I would pass into my tests. This object has one method, GetBrowser(), which calls Start() on it’s browser and passes it back:

public class SeleniumProvider
{
    private ISelenium Browser;

    public ISelenium GetBrowser()
    {
        Browser.Start();
        return Browser;
    }
}


My CreateBrowser() code was then changed thus:

private SeleniumProvider CreateBrowser()
{
    if (_url == null)
        throw new NullReferenceException(
            "No [URL] attribute was defined, either define [URL], or pass the URL of the start page to BrowserAttribute's constructor");

    switch (_browser)
    {
        case "Internet Explorer 6":
            Browser = new DefaultSelenium("testrunner1", 4444, "*iexplore", _url);
            break;
        // <SNIP>
    }
    return new SeleniumProvider(Browser);
}


So I can now happily write tests that look like this:

[Theory]
[URL("http://www.google.com")]
[Browser(Browsers.InternetExplorer7)]
[Browser(Browsers.Firefox3_5)]
[Browser(Browsers.GoogleChrome)]
[Browser(Browsers.Opera)]
public void Google_For_SimpleTalk(ISeleniumProvider seleniumProvider)
{
    Browser = seleniumProvider.GetBrowser();
    //Test code
}

At this point I was challenged by Matt Lee:

“What would be really smart would be if you could define multiple [URL]’s, and have each [Browser] test every [URL]!”

So having come this far, I decided to take on that challenge as well!

After a short period of time thinking, it became clear that I could not achieve this goal within the framework I already had. Despite knowing about all potential [URL]’s from within my BrowserAttribute code, I had no way off passing back multiple browser objects to a test method without re-introducing the kind of TestSetup-style code I was attempting to avoid. Instead, I would have to provide my own implementation of the code responsible for enumerating test data. This code lives in the TheoryAttribute, so to provide my own implementation I created the [BrowserTheory].

The code in question lives in TheoryAttribute.GetData(). This static method takes a MethodInfo, and yield returns an IEnumerable of object arrays containing the test data:

static IEnumerable<object[]> GetData(MethodInfo method)
{
foreach (BrowserAttribute attr in method.GetCustomAttributes(typeof(BrowserAttribute), false))
{
    ParameterInfo[] parameterInfos = method.GetParameters();
    Type[] parameterTypes = new Type[parameterInfos.Length];

    else foreach (URLAttribute attribute in method.GetCustomAttributes(typeof(URLAttribute), false))
    {
        attr._url = attribute.Url;
        IEnumerable<object[]> browserAttrData = attr.GetData(method, parameterTypes);
        if (browserAttrData != null)
            foreach (object[] dataItems in browserAttrData)
                yield return dataItems;
    }
}
}


Once it has constructed the array of Types required to call GetData() on each BrowserAttribute, the above code is doing one of two things. First it checks to see if the BrowserAttribute already has a URL defined. If it does, then it proceeds using that URL, ignoring any URLAttributes that may have been defined. This is to facilitate the use of the old style [Browser(string browser, string url)].

Otherwise, if there is no URL defined, it goes on to return that BrowserAttribute once for every URLAttribute it discovers attached to the test method.

That was all I needed to do! Now that the [URL]’s are being handled elsewhere, I no longer needed my BrowserAttribute.GetURLFromAttribute() method, so I removed it.

Finally, my example test case (expanded to include all browser versions) now reads like this:

[BrowserTheory]
[URL("http://www.google.co.uk")]
[URL("http://www.google.com")]
[Browser(Browsers.InternetExplorer6)]
[Browser(Browsers.InternetExplorer7, "http://www.google.com")]
[Browser(Browsers.InternetExplorer8)]
[Browser(Browsers.Firefox2)]
[Browser(Browsers.Firefox3)]
[Browser(Browsers.Firefox3_5)]
[Browser(Browsers.GoogleChrome)]
[Browser(Browsers.Opera)]
public void Google_For_SimpleTalk(SeleniumProvider seleniumProvider)
{
    Browser = seleniumProvider.GetBrowser();
    //Test code
}


I’ve included one example of the old-style [Browser], just to demonstrate how the two co-exist. The above will execute the test case twice for each browser (except Internet Explorer 7), starting from the two Google URLs. One test case execution will occur using IE7, starting from the google.com URL specifically provided to that BrowserAttribute. That makes a grand total of 15 test runs, for one copy of the test code 🙂