敏感变量和感知机结合的测试预言生成方法
|
摘要
测试预言生成技术是软件工程测试领域的研究热点之一.没有可以利用的历史测试用例是目前大部分测试预言生成技术的普遍假设,但是这种假设会使我们错过利用现有部分测试用例协助自动生成新测试用例预言的机会.在已知部分测试用例集的情况下,提出了基于敏感变量和线性感知机的新测试用例的测试预言自动生成方法.首先,收集已知测试用例执行的语句覆盖和不同断点处内存值集合作为训练集,计算与新测试用例执行覆盖信息高度相似的测试用例集;其次,计算各断点处表征成功或失败的敏感变量集;然后,应用线性感知机求解每个断点处成功或失败概率预测的门限值;在此基础上,给出新测试用例测试预言自动生成的方法,并对方法进行讨论;最后,采用7个程序的129个故障版本作为实验对象,对共计14 300个测试用例生成测试预言.实证评价表明,测试预言准确率平均达到96.2%.该成果可以形成测试用例集合构造的"滚雪球效应",不断迭代自动生成新测试用例的测试预言.
Test oracle generation technology is one of the hotspots in the testing field of software engineering. There are no historical test case sets available, which are common assumptions about existing test oracle generation techniques. However, this assumption may not always hold, and where it does not, there may be a missed opportunity; perhaps the pre-existing test cases could be used to assist the automated oracle generation of new test cases. In the case of the existing test case set, an automatic test oracle generation method for a new test case based on sensitive variables and linear perceptrons is proposed. Firstly, the statement coverage and memory value set executed by some known test cases are collected, and a set of test cases with high similarity to the execution coverage information of the new test case is computed. Secondly, the memory sensitive variable set solving algorithm of the program at a given breakpoint is given.Thirdly, the known test case set as the training set and the perceptron is used to solve the threshold value at each breakpoint. And on this base an automatic oracle generation method of the new test case is proposed. Finally, 129 fault versions of seven programs are used as experimental objects to generate test oracles of 14 300 new test cases. The empirical evaluation shows that the generated test oracle can achieve 96.2% accuracy on average. The results of the research can form the "snowball effect" of the test case set construction, and iteratively automatically generate test oracles for new test cases.
Test oracle generation technology is one of the hotspots in the testing field of software engineering. There are no historical test case sets available, which are common assumptions about existing test oracle generation techniques. However, this assumption may not always hold, and where it does not, there may be a missed opportunity; perhaps the pre-existing test cases could be used to assist the automated oracle generation of new test cases. In the case of the existing test case set, an automatic test oracle generation method for a new test case based on sensitive variables and linear perceptrons is proposed. Firstly, the statement coverage and memory value set executed by some known test cases are collected, and a set of test cases with high similarity to the execution coverage information of the new test case is computed. Secondly, the memory sensitive variable set solving algorithm of the program at a given breakpoint is given.Thirdly, the known test case set as the training set and the perceptron is used to solve the threshold value at each breakpoint. And on this base an automatic oracle generation method of the new test case is proposed. Finally, 129 fault versions of seven programs are used as experimental objects to generate test oracles of 14 300 new test cases. The empirical evaluation shows that the generated test oracle can achieve 96.2% accuracy on average. The results of the research can form the "snowball effect" of the test case set construction, and iteratively automatically generate test oracles for new test cases.
引文
[1]Barr ET,Harman M,Mcminn P,Shahbaz M,Yoo S.The oracle problem in software testing:A survey.IEEE Trans.on Software Engineering,2015,41(5):507-525.[doi:10.1109/TSE.2014.2372785]
[2]http://sir.unl.edu/portal/index.php
[3]Arantes AO,de Santiago VA,Vijaykumar NL.On proposing a test oracle generator based on static and dynamic source code analysis.In:Proc.of the IEEE Int’l Conf.on Software Quality,Reliability and Security.IEEE,2015.144-152.[doi:10.1109/QRS-C.2015.29]
[4]Goffi A.Automatic generation of cost-effective test oracles.In:Proc.of the ICSE 2014:Int’l Conf.on Software Engineering.2014.678-681.[doi:10.1145/2591062.2591078]
[5]Gholami F,Attar N,Haghighi H,Asl MV,Valueian M,Mohamadyari S.A classifier-based test oracle for embedded software.In:Proc.of the 2018 Real-Time and Embedded Systems and Technologies(RTEST).2018.104-111.[doi:10.1109/RTEST.2018.8397165]
[6]Zeller A.Yesterday,my program worked,today,it does not.Why?In:Proc.of the Joint Meeting of the European Software Engineering Conf.and the ACM SIGSOFT Symp.on the Foundations of Software Engineering(ESEC/FSE’99).1999.253-267.[doi:10.1145/318774.318946]
[7]Memon AM,Pollack ME,Soffa ML.Automated test oracles for GUIs.In:Proc.of the Century Applications.ACM Press,2000.30-39.[doi:10.1145/357474.355050]
[8]Ye M,Feng BQ,Zhu L,Lin Y.Automated test oracle based on neural networks.In:Proc.of the IEEE Int’l Conf.on Cognitive Informatics.IEEE,2007.517-522.[doi:10.1109/COGINF.2006.365539]
[9]Jin H,Wang Y,Chen NW,Gou ZJ,Wang S.Artificial neural network for automatic test oracles generation.In:Proc.of the Int’l Conf.on Computer Science and Software Engineering.IEEE Computer Society,2008.727-730.[doi:10.1109/CSSE.2008.774]
[10]Xu WF,Ding T,Wang HL,Xu DX.Mining test oracles for test inputs generated from Java bytecode.In:Proc.of the Computer Software and Applications Conf.IEEE,2013.27-32.[doi:10.1109/COMPSAC.2013.8]
[11]Murphy C,Shen K,Kaiser G.Using JML runtime assertion checking to automate metamorphic testing in applications without test oracles.In:Proc.of the 2009 Int’l Conf.on Software Testing Verification and Validation(ICST 2009).Washington:IEEEComputer Society,2009.[doi:10.1109/ICST.2009.19]
[12]Doong RK,Frankl PG.The ASTOOT approach to testing object-oriented programs.ACM Trans.on Software Engineering and Methodology,1994,3(2):101-130.[doi:10.1145/192218.192221]
[13]Li N,Offutt J.Test oracle strategies for model-based testing.IEEE Trans.on Software Engineering,2017,43(4):372-395.[doi:10.1109/TSE.2016.2597136]
[14]Guo HF.A semantic approach for automated test oracle generation.Computer Languages Systems&Structures,2016,45:204-219.[doi:10.1016/j.cl.2016.01.006]
[15]Wang F,Yao LW,Wu JH.Intelligent test oracle construction for reactive systems without explicit specifications.In:Proc.of the2011 IEEE 9th Int’l Conf.on Dependable,Autonomic and Secure Computing(DASC).IEEE,2011.89-96.[doi:10.1109/DASC.2011.39]
[16]Shahamiri SR,Kadir WM,Ibrahim S,et al.An automated framework for software test oracle.Information and Software Technology,2011,53(7):774-788.[doi:10.1016/j.infsof.2011.02.006]
[17]Shahamiri SR,Wankadir WM,Ibrahim S,Hashim SZM.Artificial neural networks as multi-networks automated test oracle,Automated Software Engineering,2012,19(3):303-334.[doi:10.1007/s10515-011-0094-z]
[2]http://sir.unl.edu/portal/index.php
[3]Arantes AO,de Santiago VA,Vijaykumar NL.On proposing a test oracle generator based on static and dynamic source code analysis.In:Proc.of the IEEE Int’l Conf.on Software Quality,Reliability and Security.IEEE,2015.144-152.[doi:10.1109/QRS-C.2015.29]
[4]Goffi A.Automatic generation of cost-effective test oracles.In:Proc.of the ICSE 2014:Int’l Conf.on Software Engineering.2014.678-681.[doi:10.1145/2591062.2591078]
[5]Gholami F,Attar N,Haghighi H,Asl MV,Valueian M,Mohamadyari S.A classifier-based test oracle for embedded software.In:Proc.of the 2018 Real-Time and Embedded Systems and Technologies(RTEST).2018.104-111.[doi:10.1109/RTEST.2018.8397165]
[6]Zeller A.Yesterday,my program worked,today,it does not.Why?In:Proc.of the Joint Meeting of the European Software Engineering Conf.and the ACM SIGSOFT Symp.on the Foundations of Software Engineering(ESEC/FSE’99).1999.253-267.[doi:10.1145/318774.318946]
[7]Memon AM,Pollack ME,Soffa ML.Automated test oracles for GUIs.In:Proc.of the Century Applications.ACM Press,2000.30-39.[doi:10.1145/357474.355050]
[8]Ye M,Feng BQ,Zhu L,Lin Y.Automated test oracle based on neural networks.In:Proc.of the IEEE Int’l Conf.on Cognitive Informatics.IEEE,2007.517-522.[doi:10.1109/COGINF.2006.365539]
[9]Jin H,Wang Y,Chen NW,Gou ZJ,Wang S.Artificial neural network for automatic test oracles generation.In:Proc.of the Int’l Conf.on Computer Science and Software Engineering.IEEE Computer Society,2008.727-730.[doi:10.1109/CSSE.2008.774]
[10]Xu WF,Ding T,Wang HL,Xu DX.Mining test oracles for test inputs generated from Java bytecode.In:Proc.of the Computer Software and Applications Conf.IEEE,2013.27-32.[doi:10.1109/COMPSAC.2013.8]
[11]Murphy C,Shen K,Kaiser G.Using JML runtime assertion checking to automate metamorphic testing in applications without test oracles.In:Proc.of the 2009 Int’l Conf.on Software Testing Verification and Validation(ICST 2009).Washington:IEEEComputer Society,2009.[doi:10.1109/ICST.2009.19]
[12]Doong RK,Frankl PG.The ASTOOT approach to testing object-oriented programs.ACM Trans.on Software Engineering and Methodology,1994,3(2):101-130.[doi:10.1145/192218.192221]
[13]Li N,Offutt J.Test oracle strategies for model-based testing.IEEE Trans.on Software Engineering,2017,43(4):372-395.[doi:10.1109/TSE.2016.2597136]
[14]Guo HF.A semantic approach for automated test oracle generation.Computer Languages Systems&Structures,2016,45:204-219.[doi:10.1016/j.cl.2016.01.006]
[15]Wang F,Yao LW,Wu JH.Intelligent test oracle construction for reactive systems without explicit specifications.In:Proc.of the2011 IEEE 9th Int’l Conf.on Dependable,Autonomic and Secure Computing(DASC).IEEE,2011.89-96.[doi:10.1109/DASC.2011.39]
[16]Shahamiri SR,Kadir WM,Ibrahim S,et al.An automated framework for software test oracle.Information and Software Technology,2011,53(7):774-788.[doi:10.1016/j.infsof.2011.02.006]
[17]Shahamiri SR,Wankadir WM,Ibrahim S,Hashim SZM.Artificial neural networks as multi-networks automated test oracle,Automated Software Engineering,2012,19(3):303-334.[doi:10.1007/s10515-011-0094-z]