• 作者：Fan Wu ; ^{fan.wu@ucl.ac.uk" class="auth_mail" title="E-mail the corresponding author} ; Jay Nanavati ^{jaysnanavati@hotmail.co.uk" class="auth_mail" title="E-mail the corresponding author} ; Mark Harman ^{mark.harman@ucl.ac.uk" class="auth_mail" title="E-mail the corresponding author} ; Yue Jia ^{yue.jia@ucl.ac.uk" class="auth_mail" title="E-mail the corresponding author} ; Jens Krinke ^{jens.krinke@ucl.ac.uk" class="auth_mail" title="E-mail the corresponding author}
• 关键词：Mutation testing ; Memory mutation
• 刊名：Information and Software Technology
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：81
• 期：Complete
• 页码：97-111
• 全文大小：898 K

Three decades of mutation testing development have given software testers a rich set of mutation operators, yet relatively few operators can target memory faults (as we demonstrate in this paper).

class="boldFont">Objective

To address this shortcoming, we introduce Memory Mutation Testing, proposing 9 Memory Mutation Operators each of which targets common forms of memory fault. We compare Memory Mutation Operators with traditional Mutation Operators, while handling equivalent and duplicate mutants.

class="boldFont">Method

We extend our previous workshop paper, which introduced Memory Mutation Testing, with a more extensive and precise analysis of 18 open source programs, including 2 large real-world programs, all of which come with well-designed unit test suites. Specifically, our empirical study makes use of recent results on Trivial Compiler Equivalence (TCE) to identify both equivalent and duplicate mutants. Though the literature on mutation testing has previously deployed various techniques to cater for equivalent mutants, no previous study has catered for duplicate mutants.

class="boldFont">Results

Catering for such extraneous mutants improves the precision with which claims about mutation scores can be interpreted. We also report the results of a new empirical study that compares Memory Mutation Testing with traditional Mutation Testing, providing evidence to support the claim that traditional mutation testing inadequately captures memory faults; 2% of the memory mutants are TCE-duplicates of traditional mutants and average test suite effectiveness drops by 44% when the target shifts from traditional mutants to memory mutants.

class="boldFont">Conclusions

Introducing Memory Mutation Operators will cost only a small portion of the overall testing effort, yet generate higher quality mutants compared with traditional operators. Moreover, TCE technique does not only help with reducing testing effort, but also improves the precision of assessment on test quality, therefore should be considered in other Mutation Testing studies.