A TDD Journey: 4-Tests as Documentation; False Positive Results; Component Isolation

In Test-Driven Development (TDD) , The writing of a unit test is done more to design and to document than to verifiy. By writing a unit test you close a number of feedback loops, and verifying the functionality of the code is just a minor one. everything you need to know about your class under test is embodied in a simple list of the names of the tests. Michael Sorens continues his introduction to TDD that is more of a journey in six parts, by discussing Tests as Documentation, False Positive Results and Component Isolation.

This is part 4 of our exploration into practicing hands-on TDD. Unlike most of my multi-part series, it is not advisable to join this one in the middle. So if you are arriving fresh, please go back and review part 1 for an overview of TDD and subsequent parts that both built out our tests & code and introduced ancillary concepts and techniques crucial to the TDD approach.

Here is a condensed version of the code so far:

2019-img2C.gif Tests as Documentation

This code has been distilled to show just what you need to understand the system as it now stands and no more. That is, if you had never seen the code before, everything you need to know is there; the missing chunks of code would only slow down your process of understanding. Consider:

Premise (from reading the tests)

Conclusion (confirmed  by reading the code)

The test class is called WidgetActivatorTest.

Hence there should be a WidgetActivator class.

WidgetActivator needs both an WidgetLoader and an IWidgetPublisher

There is an WidgetLoader interface and an IWidgetPublisher interface, and these are both injected dependencies to the WidgetActivator constructor.

We are exercising an Execute method to do loading and publishing.

The WidgetActivator has an Execute method which invokes some WidgetLoader method to load and some IWidgetPublisher method to publish.

The final test says Execute needs to return a status.

The Execute method signature shows it returns a Boolean.

A careful reading of the above will reveal that the code has only and exactly what the tests demand. The only reason WidgetActivator.Execute returns a Boolean is because a test needs it to. Contrast that with WidgetLoader.Load and IWidgetPublisher.Publish, which both return void. Later, there will be tests that require these methods return something so, at that time, they will adapt to do so.

Now watch closely as the quality of your ignorance takes a leap right here (see part 1 if you think this is an insult :-)! In the table above, discard the right-hand column: you can immediately infer all of that from the left-hand column. But that means that, correspondingly, you do not have to look at the code, only the tests! Therefore, the code I showed above is too verbose; in actuality, all you need to understand the system as it now stands is the list of test names:

From the test names you can generate the premises in the table above, and from the premises you can derive the conclusions. QED.


Recall in the last test (Execute_returns_false_if_no_details_to_load) that we simply hard-coded a false return value to make that test pass.

TEST: Now we make it more realistic by adding the opposite in a new test: if the loader did its work then Execute should return true.

This test finally reveals what you need mock objects for. After declaring the mockWidgetLoader above, the Setup/Returns methods together provide some emulated behavior for this test double, to wit: whenever the Load method is called, return true. By passing that mock object in to the WidgetActivator constructor, that mock object will be used inside the Execute method, triggering the Setup/Returns action specified. But before you can actually see that, there is a compilation error to address.

CODE: The above test fails to compile because the Load method does not return anything. Let’s change it to return a Boolean to get it to compile.

CODE: Now it compiles but the test reports failure in the Assert, because we had hard-coded the Execute method to return false. Revise the code to pass the return value of the Load method as the return value of Execute.

Because we “pre-loaded” the Load method to return true, Execute will now return true in this test case, so the test passes. Hooray! But this change should have repercussions in the previous test which, as you recall, was testing just the opposite: if Load returns false then Execute should return false. If you rerun all the tests you will see that the test still passes-even though we did not prepare the mocked Load method to take any specific action! Here is the prior test (from part 3)-notice just two stubs, no method actions.

So why does that test still pass? Moq is smart enough so that, in the absence of specific instructions, it will return false for a Boolean method. Thus when Load is called, it does in fact return false, and therefore Execute returns false, and the test passes. Neat!

TEST: Analogous to how the loader works, the publisher needs to return a status and the result of the Execute method should reflect the results of the publisher as well as the loader. Add a test to reflect the status of the publisher.

CODE: That fails to compile because the Publish method does not return anything, just like we saw with the previous test. Let’s change it to return a Boolean to get it to compile.

2019-img2C.gifBeware Passing Tests

Now the test passes so the code must be doing just what it needs to satisfy this test. A reasonable statement… but completely wrong! We have not even hooked up the output of this Publish method yet! So why does it pass?  Here is the body of the Execute method again, line by line:



var result = _widgetLoader.Load();

Set the result to be the output of Load. Because we have not directed moq to take any particular action for this method, it returns false as you just learned.


Now we call Publish… and completely ignore its output; thus completely ignoring the forced return value of false we directed moq to provide in the test.

return result;

So merely by coincidence, the value we return happens to reflect the return value of the Publish method.

Coincidence like this can happen more often than you might think or expect (i.e. getting a test result that happens to agree with your expectation due to incorrect code). Or to state that another way:

Watch out for tests that pass for the wrong reason!

When you have a failing test, you know a priori the code is not (yet) correct. But when you have a passing test, you must confirm that it is passing for the reason you expect. As you continue to add tests, you may expose the flaw in an earlier test that falsely passes, but this is not always the case. The onus is on you to understand your test and your code well enough to be able to say that, yes, the test is passing because what the test is expecting is being delivered for the reasons specified in the test.


So going back to our test at hand, we want to force the mock publisher’s Publish method to return false and we want that return value to be reflected in the return value of the Execute method. As you just saw above, we are ignoring the output of the Publish method, so the test is returning a false positive result.

CODE: Let’s now actually use the output of the Publish method and see if it still works.

Here we have adjusted the Execute method’s return value to be a conjunction, reflecting both the result of the Load and the Publish. The test still passes but now it passes for the right reason: we set up the mock object so that Publish will return false. The conjunction of that value with the Load result is still false, so Execute returns false, just as we wanted. So that test is complete. Unfortunately, we have an issue elsewhere…

2019-img2C.gifIsolating Components

Rerunning the whole test suite reveals that the previous test now fails. Why? Because it was assuming the return value of Execute solely depended on the WidgetLoader.Load method and we just invalidated that assumption by making it also depend on IWidgetPublisher .Publish. Therefore, when you have more than one component that can affect the result of your class under test, you must isolate the component you wish to test. And that is the true purpose of mock objects.

In this case it is straightforward. Execute is supposed to reflect the return values of both IWidgetLoader.Load and IWidgetPublisher .Publish. If both are successful, Execute should return true but if either or both fails, Execute should reflect that failure. Thus Execute depends on a conjunction of the two results. To isolate the result of a conjunction to depend exclusively on one of those items, just force the other item to be true. Consider the truth table for a conjunction of our sub-results:




Load ^ Publish

















First, examine rows 3 and 4 where Publish returns false. As you can see, whatever the result of Load is, the conjunction returns false. But now look at rows 1 and 2 which says that if Publish always returns true, then the result of the conjunction exactly matches the result of Load, just what we need. And we can force Publish to always return true by specifying that in the mock.


TEST: To isolate the Load method we just need to force Publish to always return true. Here we are using Linq-to-Mocks syntax to always return true whenever Publish is called.

Refer back to part 2 where we introduced Linq-to-Mocks syntax: here we are directing the stubWidgetPublisher to return true whenever the Publish method is invoked, isolating the result of Execute to depend solely on the outcome of the Load method.

That test is concerned with seeing what happens when the Load method returns true. Similarly, we want to review the prior test that examined what happens when Load returns false. Close examination reveals this test passed coincidentally. That is, now that we have hooked up the Publish method into the results, we are now getting a false result because the Publish method returns false (the default return result) regardless of what the Load method returns. So again, we isolate the Load method by forcing the IWidgetPublisher stub to return true.

By now you should be starting to have a grasp on the workflow of TDD: write some test code then write some production code to make the test pass. That is necessary but not sufficient-you must also make sure the test and the code together are doing what you think! In the next installment you will add yet more tests and pickup more practical tips to help your TDD journey.