You might have heard about Mutation Testing before. In the last 5 or 6 years it’s been a reasonably hot (“warm”?) topic to discuss in blogs and dev talks. So what is the added value over code coverage with just Unit Testing? Even if you could pride yourself with over 90% line and branch coverage, that coverage means nothing apart from that unit tests are touching production code. It says nothing about how well that code is tested, it doesn’t care whether any asserts exist in your tests. Imagine an engineer that tests a power drill he designed on a sheet of paper, and declaring that it does exactly what it was designed for: drilling holes. It’s obvious that this test is meaningless for a power drill that is meant to be used on wood, steel or stone.

Mutation tests aim to expose tests that cover the lines they’re meant to cover but are insufficient in testing the intent of the code. The idea behind this is fairly simple: introduce “mutants” in the code that is being tested, and check whether the unit tests that cover these mutants still succeed or start to fail. If a test still succeeds, that means the test falls short of verifying the complete intent of the code!

So what is a “mutant” in this case? Most of the time, it’s as simple as an operator +, -, &&, || etc., that is “mutated” into a different one. In this way, a mutation testing framework helps to identify code that is not tested sufficiently and identifies weak tests that still succeed when mutants are introduced.

If this is something you consider to be a valuable addition to your project, consider trying out Mutation Testing by following the guide below.

In this blog post, I walk through the steps of configuring and using the mutation testing framework PIT for a Java codebase and explain how to integrate it with the code quality inspection platform SonarQube.