Unit Testing - A Pragmatic Approach

There's a lot of "should dos" out there regarding unit tests- Test Driven Development (TDD) is a concept a lot of professional developers get behind with an almost religious fervor, pointing to simple examples of how to implement a test-driven method which usually involves:

• Write a test
• Make the test fail
• Write the code
• Make the test pass
• Profit.

...but really, this is completely impractical for a number of reasons:

1) Most compiled languages will require you to write the code first, otherwise the test will have nothing to actually test.  Writing a test for Foo() is great, except if Foo() doesn't exist, this will fail for the wrong reasons.  So building the test first doesn't always make a lot of sense.

2) Even simple methods have several tests that are applicable for each.  Consider the following code:


public bool Foo(string bar, int wee)
   {
   if (bar == "foo" && ((1 / wee) * 10) > 1))
      {
      return true;
      }
   else
      {
      return false;
      }
   }


How many conditions do we need to check for here, to see if the method:

• Produces the proper output
• Handles odd inputs gracefully

Let us count the ways:

1. Test string bar for empty string condition
2. Test string bar for null value condition
3. Test string bar for "foo" condition
4. Test string bar for "Foo" condition
5. Test string bar for "Something Else" condition
6. Test int wee for zero condition
7. Test int wee for negative condition
8. Test int wee for 5 condition
9. Test int wee for "not 5" condition

Why so many tests?  For a nullable input, not testing a null value is asking for trouble.  Likewise, strings can easily be empty, and often when not expected.  These are the sorts of things that unit tests are meant to help with: testing for possible but unexpected data.

For integers, testing for zero is a given.  In the method above, a value of zero will throw a DivideByZero exception.  What if we were multiplying that number against a cost value, and someone inadvertently tossed an Int32.MaxValue as an argument?  Not testing for the default value of any variable is asking for trouble.  Likewise, negative numbers... maybe the code you are using this method with has a buggy math operation, if the application bombs in Foo() because it doesn't recognize the problems inherent in other code, you have failed at testing.  Edge cases are usually a good idea too. (max and min values)

I have 9 tests listed for method Foo.  And that's normal!  This is a fairly well-tested method.  You could take some shortcuts, but ultimately, you are looking at 3-4 tests per argument at a minimum, not to mention dealing with potential combinations that need to be handled gracefully.

3) If you are in a technology-centric company, you likely have leadership that understands the importance of unit testing and quality control.  If, however, you are employed in the other 95% of businesses, your chances of having leadership that understands and appreciates this sort of effort dramatically decreases.  Technology people want accurate results; business people tend to want quick results, and the two rarely overlap.

These are the reasons I have never been a big fan of TDD... but that doesn't mean I'm not a fan of unit testing.  Unit testing is an incredibly important part of any quality software production.  However, following doctrine for how unit testing should be done has resulted in slowdowns that bosses usually do not appreciate.

Here are some ways I have made unit testing work for me and have given me a degree of confidence in the code I have produced:

• Always test for default values of your argument types.  When coding, it's easy to declare/ instantiate an object, expect it to be properly populated, and then pass it into a method... only to find out that the object was not, in fact, properly populated.  Nullable objects should be checked for null values, integers should be checked for zero, etc.  Check for the values in your method, and throw the proper exception (ArgumentNullException or OutOfRangeException) when the data needs to actually be used.  Provide enough info to understand where the error occurred.

• Keep your exceptions granular, so you can test for a specific condition, for example:


bool Foo(string foo)
{
   if (foo == null) throw new ArgumentNullException("foo");
   if (foo == string.Empty) throw new ArgumentException("parameter 'foo' cannot be empty")
...
}


          In this example, we have two easily testable scenarios:

[Test, ExpectedException(typeof(ArgumentException))]
public void testFooEmptyStringArg()
{
   var f = Foo(string.Empty);
}

[Test, ExpectedException(typeof(ArgumentNullException))]
public void testFooEmptyStringArg()
{
   var f = Foo(null);
}


• Test against interfaces as much as possible.  Internal workings of components should be tested, but the public usage of the classes and methods are the ones that will need the most scrutiny.

• Manage your time.  Remember, in a perfect world, you'd have infinite time to create the perfect code and unit tests... but development is generally funded by business, and in business, time is money.  Start with the most obvious conditions to test, get a degree of confidence, and expect to come back to troubleshoot and fix bugs.  Whenever you fix a bug, add a unit test to make sure that bug never "reappears."

• Always keep in mind the intent behind unit testing is not to create bug-free code, nor is it to provide a troubleshooting mechanism for bugs.  It is a validation tool to confirm that your code behaves the way you meant it to... and when you have to change your code base, well-written unit tests will show you what your latest changes have broken in other places, which will drastically reduce regression testing and user acceptance time.

That's a basic overview.  Comments are welcome (I expect a few will be how wrong I am)...