Monthly Archives

January 2019

Mutation testing in Maven & Sonarqube

Introduction

You might have heard about Mutation Testing before. In the last 5 or 6 years it’s been a reasonably hot (“warm”?) topic to discuss in blogs and dev talks. So what is the added value over code coverage with just Unit Testing? Even if you could pride yourself with over 90% line and branch coverage, that coverage means nothing apart from that unit tests are touching production code. It says nothing about how well that code is tested, it doesn’t care whether any asserts exist in your tests. Imagine an engineer that tests a power drill he designed on a sheet of paper, and declaring that it does exactly what it was designed for: drilling holes. It’s obvious that this test is meaningless for a power drill that is meant to be used on wood, steel or stone.

Mutation tests aim to expose tests that cover the lines they’re meant to cover but are insufficient in testing the intent of the code. The idea behind this is fairly simple: introduce “mutants” in the code that is being tested, and check whether the unit tests that cover these mutants still succeed or start to fail. If a test still succeeds, that means the test falls short of verifying the complete intent of the code!

Continue Reading