When people write tests there is one metric I hear them mention quite often. Yes, you guessed it. Code coverage is mentioned by many people when talking about how well their code is tested. I believe this is a very useful metric, but as the title of this post states, code coverage is not enough. When testing code it is important to try to check all cases which are likely to be able to produce different results. Odd and interesting cases are important.
Tests should be written to handle different cases which can occur. They should not be written to "cover the code". If you're handling the different cases you will also have code coverage. Notice the important difference here. Why don't we take a look at a nice example here.
For the sake of this example we will assume we have 2 classes; medic and soldier. Medics are supposed to heal wounded soldiers and only wounded soldiers. To check this we will make a method for the medic so it can decide.
public bool ShouldHealSoldier(Soldier soldierToCheck)
{
return soldierToCheck.IsWounded;
}
So now we will test for code coverage.
[TestMethod]
public void MedicShouldHealSoldierWhoIsWounded()
{
var s = new Soldier();
s.IsWounded = true;
var m = new Medic();
Assert.IsTrue(m.ShouldHealSoldier(s));
}
[TestMethod]
public void MedicShouldNotHealSoldierWhoIsNotWounded()
{
var s = new Soldier();
s.IsWounded = false;
var m = new Medic();
Assert.IsFalse(m.ShouldHealSoldier(s));
}
Notice that I am not really testing much here. I am just making sure to run each line of code. This tends to happen when people test for code coverage. If I were instead trying to handle the cases I thought might come up in this code I probably would have done the following tests.
[TestMethod]
public void MedicShouldHealSoldierWhoIsWounded()
{
var s = new Soldier();
s.IsWounded = true;
var m = new Medic();
Assert.IsTrue(m.ShouldHealSoldier(s));
}
[TestMethod]
public void MedicShouldNotHealSoldierWhoIsNotWounded()
{
var s = new Soldier();
s.IsWounded = false;
var m = new Medic();
Assert.IsFalse(m.ShouldHealSoldier(s));
}
[TestMethod]
[ExpectedException(typeof(ArgumentNullException))]
public void ShouldHealSoldierShouldThrowArgumentNullIfSoldierIsNull()
{
var m = new Medic();
Assert.IsTrue(m.ShouldHealSoldier(null));
}
Notice that now I am handling an extra case. I am not covering the code any more with this, because I can't get any more than 100% test coverage, which is what I had after the first tests. You will notice though that I've tested the code better than I did before. I now also know that my code doesn't handle the exception the way I want it to. It currently throws a null reference exception, but I want it to do something else. By writing the code this way I will now get the method written the way I want it to be written. Not the way I wrote it.
This is just one simple example of how adding extra tests which don't add any extra code coverage help. One big reasons to write tests is so code works the way you want it to. That sounds silly, but if you don't write the tests you will often write code that doesn't do what you want it to. This gets all of your expectations for the code out there so that you will be sure to create them all.
Comments