Test Driven Development ROI

Ever think about the true benefits of tests and the value they provide you and your company? This article basically challenges developers to consider the ROI (Return On Investment) of every test. It has had a big impact on my software development habits by making me examine the benefits vs. the costs of writing tests.

Here is one of his most interesting comments: "... the developer is concentrating on testing before "coding" - the primary function is hence the development of tests, leading to a situation of "test-oriented development. This is where the production of tests becomes the overriding concern and output of a team, with the development of required functionality being secondary. When this occurs, the design, quality, and scope of the tests all become greater than that of the system actually being created". I am a firm believer in unit tests, but I hope I never fall into that category. We have a saying here at work, "I can get you about 90% confidence, but it's going to cost a lot to get that other 10%".

Oh yeah, its now Test First Development.

Also here is a follow up article to the original link above: When TDD Goes Bad #2. In it the author explains Mock oriented development and the lack of integration tests.

How to capture logging output in unit tests

When writing unit tests for java, its extremely valuable to view the log statements produced by your unit tests and java code. The following is a brief introduction on how to configure maven1 and maven2 projects to capture logs.

Maven1
The maven-test-plugin is used for running unit tests in maven1 projects. It is best to configure this plugin at a global level in your build.properties file located under your $USER_HOME.

First add the following to your $USER_HOME/build.properties file:

maven.junit.sysproperties=log4j.configuration
log4j.configuration=file:/c:/Documents and Settings/jlorenzen/log4j.properties

Second add a log4j.properties file under your $USER_HOME directory. Mostly likely this will include a ConsoleAppender. By using a ConsoleAppender the maven-test-plugin will capture the logs and write them to the console. However its cleaner if you set the property maven.junit.usefile=true in your build.properties file. This will put all your logs in the .txt test report file.

Maven2
Unfortunately things are a bit different for maven2 projects. You no longer use the maven-test-plugin but rather the maven-surefire-plugin. The following example will use java logging.

To get started open up your project's parent pom.xml and add the following for the maven-surefire-plugin.

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemProperties>
<property>
<name>java.util.logging.config.file</name>
<value>${logging.location}</value>
</property>
</systemProperties>
</configuration>
</plugin>
</plugins>
</build>

Next you need to define the value for the variable ${logging.location}. You do this in your settings.xml file which is located under $USER_HOME/.m2. If you do not already have a settings.xml file you can get an empty one under your maven2 install in the directory $M2_HOME/conf. To define the value for ${logging.location} create a profile in your $USER_HOME/.m2/settings.xml file like the following:

<profiles>
<profile>
<id>configure.logging.location.profile</id>
<properties>
<logging.location>${user.home}/.m2/logging.properties</logging.location>
</properties>
</profile>
</profiles>

Next you need to activate the profile by adding the following in your settings.xml:

<activeprofiles>
<activeprofile>configure.logging.location.profile</activeprofile>
</activeprofiles>

Since the surefire plugin does not put log statements in the output file when using the redirectTestOutputToFile, viewing the log statements in the console can be a bit overwhelming. To make this easier consider using the test single goal: mvn -Dtest=ClassName test

How to compare XML in unit tests

When writing unit tests that compare basic XML I have historically just kept it simple and used string comparison. Granted it's highly error prone and an extra whitespace, tab, or incorrect order can cause the test to fail, but it's quick and when your test is simple enough it's sufficient. The strategy I tend to take when writing code also extends to my unit tests: start simple and improve the code as you go along. So when string comparison works; use it. When your unit test demands more then perhaps you should look at XMLUnit. It's actually quite simple and helps you avoid that nasty feeling you get when you use .equals().

Your first iteration might look something like this. Here you are expecting the method getResult() to return back '<basic/>'.

String expectedXml = "<basic/>";
String result = xmlBuilder.getResult();

assertEquals(expectedXml, result);

Now lets say getResult() returns back '<basic></basic>'. Now the test will fail even though the XML is equivalent. At first it's tempting to just add another test that expects the new format, and to be honest I seem to recall doing that. But now I have a new tool to use, so here is how you can use XMLUnit to test XML.

First start by adding it as a dependency. If you are a maven2 user add the following to your POM:

<dependencies>
   <dependency>
       <groupid>xmlunit</groupid>
       <artifactid>xmlunit</artifactid>
       <version>1.0</version>
       <scope>test</scope>
   </dependency>
</dependencies>

Next refactor your unit test to extend org.custommonkey.xmlunit.XMLTestCase instead of junit.framework.TestCase.
Finally replace the assertEquals() with assertXMLEqual() and that is it. So now your test would look like:

String expectedXml = "<basic/>";
String result = xmlBuilder.getResult();

assertXMLEqual(expectedXml, result);

And would pass for all combinations of '<basic/>'; such as '<basic></basic>'. As an additional hint you also might want to turn on ignoring whitespace. To accomplish this insert the following line in your code (I placed it in my constructor) to ignore whitespaces:

XMLUnit.setIgnoreWhitespace(true);

Now the other combinations will pass such as: '<basic>   </basic>'.

For more information on XMLUnit see their Java User's Guide.