Mutation Testing

This is basically a white box testing which is useful to verify the test cases quality. In mutation test we create a mutant program and replace them with original program, and it is expected that the test cases should catch those mutant programs. We are going to discuss how to perform mutation testing. Lets start.

Mutant programs are very similar to the original program but small changes are there which effect the output. Test cases should catch such a mutant programs.

Expectations are mutant programs should fail, as you can see the above image if we run test the original defect free program should run but if we made the mutant program then test case should catch it and text case should get failed. This case may also be vice versa means, previous failed test case for original programs should pass for mutant programs if the output is appropriate. Mutant code should not survive but if survive then there might be some problem. 

Example : 

This is original/production code. Mutant code for this production code can be as follow

Mutant code 1:

Mutant code 2:

So see we have slightly changed the code, in first mutant code instead of + we have entered * and in second code we have entered – instead of +. Mutant code should be very similar with the production code but need to make very small changes as shown in above examples. So if we are executing our text cases in manual testing or test script in automation testing then these mutant test cases/test script should get failed. This would tell us the effectiveness of the test cases and test scripts.

So in testing, first testers need to apply these test scripts / test cases to original production code. And after that testers need to apply this code to the mutant code. Once the execution is done on both code (production code and mutant code) then testers verify and compare the result. Mutant code should be identified and killed by the test cases. If both the result are same means the result of production code and the result of the mutant code then will can say that mutant is alive and we need to improve the effectiveness of the test cases so that it will be effective and would kill the mutant but the test result are different then we can say that our test scripts or test cases are good enough and effective to kill the mutant.

To generating mutant code, we can do that manually or we also have some tool which will perform the task for us and create the mutant code. One of the tool is PIT, this tool create a mutant code my making some code changes. For example, it can change the > into the =< or >=, it can change = into != , - to + or * or / , etc it can make these type of slight changes in the code to produce the mutant code. These PIT tool can be Ant, command line or with maven. Otherwise we could do these changes manually also.