Code Contracts: Unit testing contracted code

Code contracts and unit tests are not replacements for each other. They both have different purpose and different nature. It does not matter if you are using code contracts or not – you still have to write tests for your code. In this posting I will show you how to unit test code with contracts.

In my previous posting about code contracts I showed how to avoid ContractExceptions that are defined in code contracts runtime and that are not accessible for us in design time. This was one step further to make my randomizer testable. In this posting I will complete the mission.

Problems with current code

This is my current code.

public class Randomizer
{
    public static int GetRandomFromRangeContracted(int min, int max)
    {
        Contract.Requires<ArgumentOutOfRangeException>(
            min < max,
            "Min must be less than max"
        );
 
        Contract.Ensures(
            Contract.Result<int>() >= min &&
            Contract.Result<int>() <= max,
            "Return value is out of range"
        );
 
        var rnd = new Random();
        return rnd.Next(min, max);
    }
}

As you can see this code has some problems:

• randomizer class is static and cannot be instantiated. We cannot move this class between components if we need to,
• GetRandomFromRangeContracted() is not fully testable because we cannot currently affect random number generator output and therefore we cannot test post-contract.

Now let’s solve these problems.

Making randomizer testable

As a first thing I made Randomizer to be class that must be instantiated. This is simple thing to do. Now let’s solve the problem with Random class. To make Randomizer testable I define IRandomGenerator interface and RandomGenerator class. The public constructor of Randomizer accepts IRandomGenerator as argument.

public class Randomizer
{
    public static int GetRandomFromRangeContracted(int min, int max)
    {
        Contract.Requires<ArgumentOutOfRangeException>(
            min < max,
            "Min must be less than max"
        );
 
        Contract.Ensures(
            Contract.Result<int>() >= min &&
            Contract.Result<int>() <= max,
            "Return value is out of range"
        );
 
        var rnd = new Random();
        return rnd.Next(min, max);
    }
}

As you can see this code has some problems:

• randomizer class is static and cannot be instantiated. We cannot move this class between components if we need to,
• GetRandomFromRangeContracted() is not fully testable because we cannot currently affect random number generator output and therefore we cannot test post-contract.

Now let’s solve these problems.

Making randomizer testable

As a first thing I made Randomizer to be class that must be instantiated. This is simple thing to do. Now let’s solve the problem with Random class. To make Randomizer testable I define IRandomGenerator interface and RandomGenerator class. The public constructor of Randomizer accepts IRandomGenerator as argument.

public interface IRandomGenerator
{
    int Next(int min, int max);
}
 
public class RandomGenerator : IRandomGenerator
{
    private Random _random = new Random();
 
    public int Next(int min, int max)
    {
        return _random.Next(min, max);
    }
}

And here is our Randomizer after total make-over.

public class Randomizer
{
    private IRandomGenerator _generator;
 
    private Randomizer()
    {
        _generator = new RandomGenerator();
    }
 
    public Randomizer(IRandomGenerator generator)
    {
        _generator = generator;
    }
 
    public int GetRandomFromRangeContracted(int min, int max)
    {
        Contract.Requires<ArgumentOutOfRangeException>(
            min < max,
            "Min must be less than max"
        );
 
        Contract.Ensures(
            Contract.Result<int>() >= min &&
            Contract.Result<int>() <= max,
            "Return value is out of range"
        );
 
        return _generator.Next(min, max);
    }
}

It seems to be inconvenient to instantiate Randomizer now but you can always use DI/IoC containers and break compiled dependencies between the components of your system.

Writing tests for randomizer

IRandomGenerator solved problem with testing post-condition. Now it is time to write tests for Randomizer class.

Writing tests for contracted code is not easy. The main problem is still ContractException that we are not able to access. Still it is the main exception we get as soon as contracts fail. Although pre-conditions are able to throw exceptions with type we want we cannot do much when post-conditions will fail. We have to use Contract.ContractFailed event and this event is called for every contract failure.

This way we find ourselves in situation where supporting well input interface makes it impossible to support output interface well and vice versa. ContractFailed is nasty hack and it works pretty weird way. Although documentation sais that ContractFailed is good choice for testing contracts it is still pretty painful.

As a last chance I got tests working almost normally when I wrapped them up. Can you remember similar solution from the times of Visual Studio 2008 unit tests? Cannot understand how Microsoft was able to mess up testing again.

[TestClass]
public class RandomizerTest
{
    private Mock<IRandomGenerator> _randomMock;
    private Randomizer _randomizer;
    private string _lastContractError;
 
    public TestContext TestContext { get; set; }
 
    public RandomizerTest()
    {
        Contract.ContractFailed += (sender, e) =>
        {
            e.SetHandled();
            e.SetUnwind();
 
            throw new Exception(e.FailureKind + ": " + e.Message);
        };
    }
 
    [TestInitialize()]
    public void RandomizerTestInitialize()
    {
        _randomMock = new Mock<IRandomGenerator>();
        _randomizer = new Randomizer(_randomMock.Object);
        _lastContractError = string.Empty;
    }
 
    #region InputInterfaceTests
    [TestMethod]
    [ExpectedException(typeof(Exception))]
    public void GetRandomFromRangeContracted_should_throw_exception_when_min_is_not_less_than_max()
    {
        try
        {
            _randomizer.GetRandomFromRangeContracted(100, 10);
        }
        catch (Exception ex)
        {
            throw new Exception(string.Empty, ex);
        }
    }
 
    [TestMethod]
    [ExpectedException(typeof(Exception))]
    public void GetRandomFromRangeContracted_should_throw_exception_when_min_is_equal_to_max()
    {
        try
        {
            _randomizer.GetRandomFromRangeContracted(10, 10);
        }
        catch (Exception ex)
        {
            throw new Exception(string.Empty, ex);
        }
    }
 
    [TestMethod]
    public void GetRandomFromRangeContracted_should_work_when_min_is_less_than_max()
    {
        int minValue = 10;
        int maxValue = 100;
        int returnValue = 50;
 
        _randomMock.Setup(r => r.Next(minValue, maxValue))
            .Returns(returnValue)
            .Verifiable();
 
        var result = _randomizer.GetRandomFromRangeContracted(minValue, maxValue);
 
        _randomMock.Verify();
        Assert.AreEqual<int>(returnValue, result);
    }
    #endregion
 
    #region OutputInterfaceTests
    [TestMethod]
    [ExpectedException(typeof(Exception))]
    public void GetRandomFromRangeContracted_should_throw_exception_when_return_value_is_less_than_min()
    {
        int minValue = 10;
        int maxValue = 100;
        int returnValue = 7;
 
        _randomMock.Setup(r => r.Next(10, 100))
            .Returns(returnValue)
            .Verifiable();
 
        try
        {
            _randomizer.GetRandomFromRangeContracted(minValue, maxValue);
        }
        catch (Exception ex)
        {
            throw new Exception(string.Empty, ex);
        }
 
        _randomMock.Verify();
    }
 
    [TestMethod]
    [ExpectedException(typeof(Exception))]
    public void GetRandomFromRangeContracted_should_throw_exception_when_return_value_is_more_than_max()
    {
        int minValue = 10;
        int maxValue = 100;
        int returnValue = 102;
 
        _randomMock.Setup(r => r.Next(10, 100))
            .Returns(returnValue)
            .Verifiable();
 
        try
        {
            _randomizer.GetRandomFromRangeContracted(minValue, maxValue);
        }
        catch (Exception ex)
        {
            throw new Exception(string.Empty, ex);
        }
 
        _randomMock.Verify();
    }
    #endregion

Although these tests are pretty awful and contain hacks we are at least able now to make sure that our code works as expected. Here is the test list after running these tests.

Conclusion

Code contracts are very new stuff in Visual Studio world and as young technology it has some problems – like all other new bits and bytes in the world. As you saw then making our contracted code testable is easy only to the point when pre-conditions are considered. When we start dealing with post-conditions we will end up with hacked tests. I hope that future versions of code contracts will solve error handling issues the way that testing of contracted code will be easier than it is right now.