Acceptance Testing with FitNesse: Symbols, Variables and Code-behind Styles

Although FitNesse can be used as a generic automated testing tool for both applications and databases, it excels at unit testing since it is designed with a Wiki-style interface that makes it very easy to set up tests. In part 5, Michael Sorens explains how to use symbols and variables effectively, and why those terms are themselves rather problematic

Contents

FitNesse is a wiki-based framework for writing acceptance tests for software systems. If you are not familiar with FitNesse, Part 1 of this series walks through a complete .NET example from writing the test in your browser to writing the C# code-behind. While FitNesse provides a rather nifty and user-friendly way to write acceptance tests in general, in practice there are plenty of quirks and glitches to watch out for. This and the subsequent parts of this series provide “tips from the trenches”, i.e. an accumulation of tips collected from intensive use of FitNesse on a daily basis to alleviate or avoid many of those pain points.

Here is your roadmap to the series, showing where you are right now:

 

Part 1: FitNesse Introduction and Walkthrough

 

Part 2: Documentation and Infrastructure

 

Part 3: Naming and Layout

 

Part 4: Debugging, Control Flow and Tracing

 

Part 5: Symbols, Variables, and Code-Behind Style

 

Part 6: Multiplicities and Comparisons

 

Part 7: Database Fixtures, Project Overview

Most sections in this article have references with actual hyperlinks to the FitNesse, fitSharp, or DbFit reference material. Some also have references to the sample test suite accompanying this series of articles, e.g. CleanCode.ConceptNotes.LayoutShowingEmbeddedNewlines. That path refers to a page on your FitNesse server. Thus if you are running on port 8080 on your local machine, the full URL to visit that page would be:
http://localhost:8080/CleanCode.ConceptNotes.LayoutShowingEmbeddedNewlines

Symbols

Storing and Retrieving Values in Symbols

FitNesse symbols work much like variables in a conventional programming language. In any test table cell you can store its value into a symbol or you can retrieve a previously stored symbol and use it as the value for the cell.

To store a cell’s value into a symbol for later reuse, use the double-greater-than (>>) operator. Notice how at runtime FitNesse displays the value that it is storing in the symbol. In this case, the Result value is not a testable condition, however; it is just storing the value.

1925-finesseimg59.jpg
!|Echo |
|Value |Result? |
|25 |>>MyValue|
1925-finesseimg5A.jpg

Echo

Value

Result?

25

>>MyValue

1925-finesseimgF.gif

Echo

Value

Result?

25

>>MyValue 25

 

To retrieve a symbol’s value for consumption, use the double-less-than (<<) operator. Here the value in the symbol is retrieved and compared against the expected value of 25. The execution color codes the Result because here we are using a standard test assertion.

1925-finesseimg59.jpg !|Echo | |Value |Result? | |<<MyValue|25 |
1925-finesseimg5A.jpg
Echo
Value Result?
<< MyValue 25
1925-finesseimgF.gif
Echo
Value Result?
<< MyValue 25 25

 

When using DbFit (to retrieve database values), you have further options. The symbol operators just shown cannot be combined with anything else within a cell; i.e. the operator and symbol must be alone in a cell. Thus there would be no way to construct a parameterized query like this:

SELECT * FROM Products WHERE Price = <<TargetPrice

DbFit provides an alternate access mechanism that does allow you to access a symbol when it is alone in a cell:

SELECT * FROM Products WHERE Price = @TargetPrice

This @symbolName notation may be used within DbFit’s Query method. (I also instrumented the DbClean method in my fixture library to honor that parameter notation as well, discussed later.)

Furthermore, DbFit also provides an alternate storage mechanism in the form of the SetParameter method, e.g.:

|Set Parameter|ExpectedCountOfRecords|10|

This provides a more concise way to store a value into a symbol than my Echo fixture does.

Reference: Symbols, Working with Parameters

Avoid Global Scoping of Symbols

Symbols are scoped not just within a test but globally across your current execution context. Remember that FitNesse symbols are like variables in a conventional language? The issues described in the classic paper Global Variables Considered Harmful apply equally strongly here.

First, consider how global scope applies. Once a symbol is defined it is available throughout the current execution context, be that one test page or an entire suite. Thus, as the example shows you can define a value in one test and reference it in a later test. (FitNesse runs tests at each depth alphabetically so the order of execution is well defined.) As long as you run a suite that is a common parent of both tests you get the results shown. But if you run the second page by itself or in a suite that does not include the first page, the value will be undefined!

 

On First Test Page

On Second Test Page

1925-finesseimg59.jpg !|Echo | |Value |Result? | |25 |>>MyValue| !|Echo | |Value |Result? | |<<MyValue|25 |
1925-finesseimg5A.jpg
Echo
Value Result?
25 >>MyValue

Echo
Value Result?
<< MyValue 25

 

1925-finesseimgF.gif
Echo
Value Result?
25 >>MyValue 25

Echo
Value Result?
<< MyValue 25 25

 

Variables

Avoid Duplicating Data (Even if it is Nearby)

Assume, for example, you need to create a path, but you also need access to specific parts of that path for later use, so you define each part you need to access. FitNesse lets you easily create nested definitions because you can use any of the three common bracket sets interchangeably: () {} []

  With Duplicated Data Without Duplicated Data
1925-finesseimg59.jpg
!define BaseName (testfile)
!define FileName (testfile.xls)
!define FilePath (\foo\bar\testfile.xls)

!define BaseName (testfile)
!define FileName (${BaseName}.xls)
!define FilePath (\foo\bar\${FileName})

Defining a variable certainly looks like defining a constant in a conventional language. But do not be misled-not only can you update the value of a variable (with a later !define statement) but when you have compound definitions like those shown above the references are dynamic. That is, if you later change BaseName to be “otherfile” then FileName and FilePath reflect this new value when you reference them! In that sense, variables are more like macro definitions than constants, per se.

Reference: Variables

Avoid Duplicating Data from a Database

The last section showed how to avoid duplication when it was easy to do so-you had everything you needed immediately available. But even if your values are tucked away in a database it is still worth the effort to avoid duplication. Consider the example below. On the left, you define an ID for a particular database record. You then also define the corresponding invoice number, diligently adding a comment that they must be kept in sync-not good enough! That makes the test very fragile. Rather, fetch the value corresponding to the key from the database, as shown on the right, storing it in a symbol.

  With Duplicated Data Without Duplicated Data
1925-finesseimg59.jpg
!define IdInvoice (12)
# This must match the record for the key value above!
!define InvoiceNumber (inv-2532)

!define IdInvoice (12)

!|query|!-SELECT InvoiceNumber
FROM Invoices
-!WHERE IDInvoice = ${IdInvoice}|
|InvoiceNumber? |
|>>InvoiceNumber |

Avoid Duplicating Data from Code

If you need to use values already defined in your code, resist the urge to duplicate the value in your FitNesse test– even if you provide a disclaimer (example, left side) ! Rather, retrieve the value from your codebase. If it is a constant in your code base create a simple fixture that exposes that constant to FitNesse. If it is in a standard .NET configuration file, you can leverage the AppSettingsFixture in my CleanCode fixture library (but note this only works for AppSettings, not ApplicationSettings!). In the example, right side, you can see how to retrieve a setting named DemoWorkingPath into the WorkingRoot symbol from the CleanCodeFixtures.dll.config file.

  With Duplicated Data Without Duplicated Data
1925-finesseimg59.jpg
# must match value in app.config!
!define WorkingRoot (\\sys1\files\test\TestA)

!|AppSettings |
|DllName |PropertyName |PropertyValue?|
|CleanCodeFixtures.dll|DemoWorkingPath|>>WorkingRoot|

Reference: AppSettings vs ApplicationSettings

Avoid Duplicating Data by Composition

The above several sections have shown how to collect data from specific sources in isolation to avoid duplication. You can combine those techniques to compose values from disparate sources as needed. One convenient way is with the Concat fixture from my fixture library attached to this article. Here is an example using a locally defined variable (IdInvoice) and a value retrieved from a database (InvoiceNumber):

1925-finesseimg59.jpg
!define IdInvoice (12)

!|query|SELECT InvoiceNumber from Invoices WHERE IDInvoice = ${IdInvoice}|
|Invoice Number? |
|>>InvoiceNumber |

!|Concat |
|Separator|Value1 |Value2 |Result? |
|_X_ |${IdInvoice}|<<InvoiceNumber|>>Description|

1925-finesseimg5A.jpg
query SELECT InvoiceNumber from Invoices WHERE IDInvoice = 12
Invoice Number
>> Invoice Number

Concat
Separator Value1 Value2 Result?
_X_ 12 <<Invoice Number >>Description
1925-finesseimgF.gif
query SELECT InvoiceNumber from Invoices WHERE IDInvoice = 12
Invoice Number
>> Invoice Number 25

Concat
Separator Value1 Value2 Result?
_X_ 12 << Invoice Number 25 << Description 12_X_25

Concat takes up to 9 values; use it to combine variables, symbols, database values, Windows-defined environment variables, and FitNesse-defined environment variables. Environment variables of either type are easily accessible within a test using standard variable notation, e.g. ${COMPUTERNAME} or the FitNesse-defined ${PAGE_NAME}.

Reference: Variables

Specify Global Variables in the Right Place

Earlier you learned that you should avoid globally-scoped symbols because those are like conventional variables and everyone knows global variables are bad. But remember to FitNesse the term variables really just means macros and everyone knows global macros are good. Got it?

So FitNesse global variables, which are OK to use, should be in your SetUp page; even the top-level SetUp page is fine if those values are needed in many tests. But do not put them in your SuiteSetUp page as logically tempting as that may seem! Here is why-with one variable in each (i.e. one in a SetUp page and one in a SuiteSetUp page) I ran this test:

1925-finesseimg59.jpg
!|Echo |
|Value |Result?|
|${SuiteSetupVariable}|abc |
|${SetupVariable} |123 |
1925-finesseimg5A.jpg
Echo
Value Result?
abc abc
123 123
1925-finesseimgF.gif
Echo
Value Result?
undefined variable: SuiteSetupVariable
abc expected
---------------
undefined variable: SuiteSetupVariable actual
---------------
Length expected 3 was 38
123 123

 

 

Notice how the render output looked perfectly correct, but at runtime it blew up. I am rather sure this is a bug so it may well be fixed in the latest FitNesse download.

Reference: CleanCode.ConceptNotes.BuggyVariablesOnSuitePage

Code-Behind Style

Use Proper Types

When you want a yes/no answer from a fixture, it is tempting to use a String instead of a Boolean return type in your fixture code just to allow for flexibility of returning an error message (below, left). While the test table looks reasonable, a failure report is often inappropriate. If you get false when expecting true, it simply does a string comparison, pointing out that the actual result was 5 characters when you were expecting 4-that makes the failure much more opaque; the length of true vs. false is really not the point! The proper way is to return a Boolean and let exceptions happen (or throw an exception if appropriate). You can then use a test tables like that at the right, trapping exceptions:

  Testing with "Overloaded" Types Testing with Proper Types
1925-finesseimg59.jpg
!|Test |
|Input|Result? |
|a |True |
|b |False |
|c |error message|

!|Test |
|Input|Result? |
|a |True |
|b |False |
|c |exception[ArgumentException] |
|c |exception["exception message"]|

Prefer Properties over Fields

FitNesse lets you use public fields or publicly settable properties for input; public fields, publicly readable properties, or public methods for output. But just as good encapsulation style in C# code dictate using properties over fields, you should observe the same practice in a FitNesse test. A second compelling reason to do so is that you get another form of strong typing. If you use fields (left side in the example) there is nothing preventing you from inadvertently mixing up what is an input and what is an output. If you use properties, you can define output properties to have a private setter so they cannot accidentally be misused as inputs.

  With Fields With Properties
1925-finesseimg59.jpg
// inputs
public int IdBankClient;
public string BankReferences;
public string IncentiveInvoices;

// outputs
public string Details;
public int IdMatch;


// inputs
public int IdBankClient { get; set; }
public string BankReferences { get; set; }
public string IncentiveInvoices { get; set; }

// outputs
public string Details { get; private set; }
public int IdMatch { get; private set; }

Discard Superfluous DLLs

A very short item, but useful for economy’s sake. When you write fixtures, you need to add references to fit.dll, fitSharp.dll, and possibly dbfit.dll. In the process of adding these references in Visual Studio the Copy Local property defaults to true. But there is no need to copy these standard libraries with your custom library, so just set Copy Local to false.

More to Come…

Part 6 delves into the nuances of multiple inputs vs. multiple outputs, multiple rows vs. multiple columns, as well as things that can trip you up when attempting to validate a value.