Acceptance Testing with FitNesse: Documentation and Infrastructure

FitNesse is a popular general-purpose wiki-based framework for writing acceptance tests for software systems, including databases. It is intended to be easy for the tester to learn and use. In this article, Michael Sorens gives a 'view from the trenches' of Fitnesse's documentation and infrastructure.

Contents

FitNesse is a wiki-based framework for writing acceptance tests for software systems. If you are not familiar with FitNesse, then you’ll find Part 1 of this series, ‘Acceptance Testing With FitNesse, The Overview‘, useful because it walks through a complete .NET example from writing the test in your browser to writing the C# code-behind. Although FitNesse provides a rather nifty and user-friendly way to write acceptance tests in general there are, in practice, plenty of quirks and glitches to watch out for. This article, and the subsequent parts of this series, provides “tips from the trenches”, by which I mean an accumulation of tips collected from intensive daily use of FitNesse to alleviate or avoid many of those pain-points.

 

 

Part 1: FitNesse Introduction and Walkthrough

 

Part 2: Documentation and Infrastructure

 

Part 3: Naming and Layout

 

Part 4: Debugging, Control Flow, and Tracing

 

Part 5: Symbols, Variables, and Code-Behind Style

 

Part 6: Multiplicities and Comparisons

 

Part 7: Database Fixtures, Project Overview

This article covers various issues with the documentation and infrastructure; subsequent parts will cover tips about such things as naming, debugging, control flow, layout, variables, comparisons, tracing and database.

Documentation

FitNesse was originally designed for testing Java code, but it has since evolved to handle .NET code with the separate fitSharp component and does so pretty well. Database support came later with the DbFit component. Those two components, together with the base FitNesse component, all provide their own documentation with different styles, organization and comprehensiveness. As so often with software documentation, it can be a challenge to find the information you are looking for. Here then, is a collection of the most useful links within the documentation of FitNesse, fitSharp, and DbFit:

The first part of this series was just intended to help get you up to speed if you had never been exposed to FitNesse, so it is more tutorial than reference. This and subsequent parts, however, lean more towards reference than tutorial. The main FitNesse site provides some additional tutorial material-notably their One-Minute Description and Two-Minute Example, among other good learning material. Use the User Guide link above to see those.

Infrastructure

Differentiating Java from .NET

By default, FitNesse works for the Java environment. To work in a .NET environment you must download the fitSharp DLL and tell FitNesse to use that instead. Typically you do this on your root page-accessible from a link at the bottom of every FitNesse page (the address is http://localhost:port/root). Use the !define directive as follows:

1847-img1D.jpg


!define COMMAND_PATTERN {%m -r fitnesse.fitserver.FitServer,fitsharp\fit.dll %p}
!define TEST_RUNNER {fitsharp\Runner.exe}
!define PATH_SEPARATOR {;}

1847-img1E.jpg

variable defined: COMMAND_PATTERN=%m -r fitnesse.fitserver.FitServer,fitsharp\fit.dll %p
variable defined: TEST_RUNNER=fitsharp\Runner.exe
variable defined: PATH_SEPARATOR=;

References: Defining Common Actions (see chapter 7), Customizing Test Execution, Using FitNesse

Include .NET-Equivalent References

You must include references in FitNessse with the !path directive to access the namespaces and classes in a DLL.This is In much the same way as a Visual Studio project. You do not need to include either FitNesse, as this is handled by the JAR file you launched, or fitSharp, handled by the TEST_RUNNER definition described in the previous section. You would, however, include DbFit if you are using database fixtures as well as your custom DLLs:

1847-img1D.jpg


!path fitsharp\dbfit.dll
!path fitsharp\dbfit.sqlserver.dll
!path Fixtures\CleanCodeFixtures.dll

1847-img1E.jpg


classpath: fitsharp\dbfit.dll
classpath: fitsharp\dbfit.sqlserver.dll
classpath: Fixtures\CleanCodeFixtures.dll

Just as with the COMMAND_PATTERN and TEST_RUNNER in the previous section, you typically specify these on your root page, which is accessible from a link at the bottom of every FitNesse page (the address is http://localhost:port/root). You may optionally specify these from a suite configuration file instead. If you do, you do not need the %p in the COMMAND_PATTERN.

Reference: Defining Common Actions (see chapter 7), Using FitNesse, Suite Configuration File

Include .NET-Equivalent Using Statements

In much the same way that you optionally import namespaces with using statements in C# to obviate the need to use fully-qualified paths when referring to classes, you may do the same in FitNessse with the !import directive. Typically these go in a SuiteSetUp or SetUp page so they are inherited as needed, but you may use them directly on individual test pages as well.

A FitNesse import is analogous to a .NET using statement: you can then write unqualified class names in your code, making it more readable. At the left side of the table below, you see the use of an !import directive allowing the Echo fixture to be referenced without a fully-qualified namespace. At right, the namespace is required when the !import directive is not used.

 

With !import

Without !import

1847-img1D.jpg


!|import |
|CleanCodeFixtures.Common|

!|Echo |
|Value|Result?|
|25 | |
|-99 | |


!|CleanCodeFixtures.Common.Echo|
|Value |Result? |
|25 | |
|-99 | |

1847-img1E.jpg

import

CleanCodeFixtures.Common

 

Echo

Value

Result?

25

 

-99

 

CleanCodeFixtures.Common.Echo

Value

Result?

25

 

-99

 

Reference: Import Fixture

Let FitNesse Dynamically Index Your Sub-Pages

You can explicitly list a set of links to subpages, as is done on the FitNesse top-level page:

1847-img1D.jpg


[[A One-Minute Description][FitNesse.UserGuide.OneMinuteDescription]]
[[A Two-Minute Example][FitNesse.UserGuide.TwoMinuteExample]]
[[User Guide][FitNesse.UserGuide]]
[[Acceptance Tests][FitNesse.SuiteAcceptanceTests]]
[[Release Notes][FitNesse.ReleaseNotes]]

1847-img1E.jpg


A One-Minute Description
A Two-Minute Example
User Guide
Acceptance Tests
Release Notes

That is fine if your list of links is hand-picked, but in most cases you will be setting up suites of suites of suites. You want each suite page to contain links to all of its children and, should you add or delete children, you want the page to update automatically. To do this, rather than enumerate your pages or suites explicitly, just use the !contents directive. The -R flag indicates to recursively show all child pages, as shown in the example. Omit the -R flag to show only immediate children. Limit the recursion to a set depth by adding a numeric argument, e.g. -R2.

1847-img1D.jpg

!contents -R

1847-img1E.jpg

Sub Suite A

  • Page A
  • Page B

Sub Suite B

  • Page A
  • Page B
  • Page C

By default, creating a new page gives you that flag and more: !contents -R2 -g -p -f -h

See the reference for details on all the other flags.

References: MarkupContents

Allow Concurrent Test Access

Where a test may access a common resource such as a database, you can allow several people to run the same test simultaneously by generating unique values per session. This assumes that each user runs their own FitNesse server, so each will thereby have a unique session.

First, setup a command file to launch FitNesse (e.g. LaunchFitNesse.cmd) defining an environment variable containing a random seed just before you launch the FitNesse server:

Windows environment variables such as SEED are accessible within FitNesse tests just like variables you explicitly define. You can thus use this session-specific seed when creating one or more values to insert in your database table:

1847-img1D.jpg


!define TestClient (MyTestClient_${SEED})
!|Echo |
|Value |Result?|
|${TestClient}| |

1847-img1E.jpg

variable defined: TestClient=MyTestClient_${SEED}

Echo

Value

Result?

MyTestClient_22861

 

From the variable definition, it does not look like the value of SEED has been substituted within TestClient, but when it is used in the Echo table you can see it has.

Avoid Duplicating Setup or Teardown Code

Use appropriate setup and teardown pages. These will function exactly as do setup and cleanup pages in unit test frameworks. The table shows the corresponding elements between the two major .NET test frameworks and FitNesse:

 

Setup test

Cleanup test

Setup suite

Cleanup suite

NUnit

[SetUp]

[TearDown]

[TestFixtureSetUp]

[TestFixtureTearDown]

MSTest

[TestInitialize]

[TestCleanUp]

[ClassInitialize]

[ClassCleanup]

FitNesse

SetUp

TearDown

SuiteSetUp

SuiteTearDown

Pages that you name SetUp or TearDown are executed automatically at the start and end of every test. Pages that you name SuiteSetUp or SuiteTearDown are executed automatically at the start and end of a suite. Note that a suite may be explicit-a page containing one or more child tests-but it may also be implicit: executing just a single test constitutes a suite, too.

These special pages are inherited in child pages, so often you just need to define these once at the top-level. However, you can override inheritance by redefining one further down your tree. This is because FitNesse pages are always arranged hierarchically in a standard tree structure. The root may typically contains suites, which may contain suites of their own, etc. Eventually, you will have leaf nodes-individual tests.

You can also inherit and override at the same time. Assume you have created a SetUp page at your root. Somewhere lower in your tree you also create a SetUp page. Its presence cancels inheritance. But if your goal is to add to the parent SetUp rather than replace it, simply start your lower SetUp page with an !include directive. My root page is CleanCode (in a browser, this is http://localhost:port/CleanCode) so referencing my root SetUp page is done like this:

1847-img1D.jpg

!include -setup .CleanCode.SetUp

1847-img1E.jpg

Included page: .CleanCode.Setup (edit)

Reference: Special Pages, Defining Common Actions (see chapter 7).

Avoid Duplicating Any Other Code, Too

You are not restricted from using the !include directive from the last section just in SetUp or TearDown pages. Bring in other needed code fragments with the !include directive whenever you need:

A path may be relative or absolute. A path address is formed just like a file system path but you must use a period between component names instead of virgules or backslashes. A leading period makes a path absolute. If your path is incorrect, FitNesse will tell you immediately upon saving the page; you do not have to wait until you execute the test.

By default, the included block of text is expanded-you see the contents of the included file inline. Compare that to the example in the previous section; the -setup option collapsed the block so you only saw the file name. The !include directive provides several options:

  • Collapse it with the -c flag.
  • Expand it and make it look just as if it was part of the page with the -seamless flag.
  • Delegate control over whether it is expanded or collapsed to the global variable COLLAPSE_SETUP or COLLAPSE_TEARDOWN with the -setup or -teardown flags, respectively.

Reference: Includes & Informational, Global Variables

Moving, Renaming, or Deleting Tests

For your conventional code-e.g. C# code that you work with in Visual Studio-there are source control plugins available (notably AnkhSVN or VisualSVN) so that, when you move or rename a file in Visual Studio, it actually mirrors the operation correctly in source control, maintaining the history of the item in question. In FitNesse, you can similarly move or rename test pages, but there is no plugin to mirror this in your source control back-end! Thus, a moved item will appear as unversioned and its former named item will appear as missing when you review your changes to commit. Because of this, it is easier to do file reorganizations outside of FitNesse, just using Windows Explorer or equivalent. Assuming you have followed the guideline above to automatically generate your suite pages with the !contents directive, the only thing you need do is restart your FitNesse server. If you don’t restart, you will pick up the items in the new locations or new names, but you will also still see the old locations/names.

Of course, the above applies to files that you have already committed to source control. If you are developing new test pages that are not yet committed, you can freely move, rename, or delete as needed within the confines of FitNesse.

More to Come…

There are many more aspects to review and issues to alleviate-stay tuned for part three!