I had the pleasure to listen very interesting podcast where Scott Hanselman and Quetzal Bradley discussed unit testing named “Testing after Unit Tests“. To increase the Google Karma of the information discussed I have decided to rehash and sum up few main ideas of this discussion.
Quetzal introduced the idea of “negative coverage” – meaning that if you have 88 % test coverage, you should not see this as something positive you have achieved, but as something missing (you miss 12 % of completion) – thus the title of this post.
He also offered very nice mental model of unit tests: see all possible paths through your code as network of roads in a state. Your unit tests are like taking small car and start driving through these roads. If you go through 88% of the roads, you can at least tell that all these roads and bridges are finished and passable with the small car driving at normal speed, that there are no fallen trees on the road and so on.
You cannot say anything about the remaining 12 % roads and bridges you have not visited. They may not exist at all – or be only half finished. You simply do not know.
It is also important to understand that you do not now how many those 88% tested roads will still be OK when you try to drive through with a 18 wheeler truck – or at 250 km/h in a sports car (feel free to fill in your favorite racing car make …).
The parallel goes even further in matching the “freeways” and side roads with main execution paths vs sideroads (you have to take freeway in order to get to the sideroad) – but to get all, get a podcast, load it to your iPod (or Zune ;-)) to find out more.
I keep listening to Hanselminutes because I like the content and the style and also to keep a tie to Microsoft world during the periods I am working outside of .NET (like in last 10 months). But even if you are not developing in Microsoft environment, many episodes of Hanselminutes very often bring ideas with much wider applicability. This is one of them.