基于弱变异准则的测试用例约简方法
|
摘要
针对为数较多的测试用例增加了回归测试成本的问题,提出一种基于弱变异准则的测试用例约简方法。首先,基于弱变异准则获得测试用例和变异分支关系矩阵;然后,重复约简4种无效测试需求和子集测试用例;最后,结合人工鱼群算法选择当前最优测试用例,并且交替执行简化和测试用例选择操作直至覆盖所有测试需求。该方法针对6个经典程序与贪心算法和HGS算法相比,基于弱变异准则并且不改变或稍微改变变异评分的条件下,约简率分别提高了73. 4%和8. 2%,且耗时分别降低了25. 3%和56. 1%。实验结果表明,所提方法在回归测试中可有效约简测试用例,降低测试代价。
In view of the problem that the test cost is increased by a large number of test suites in regression testing,a test suite reduction method based on weak mutation criterion was proposed.Firstly,the relation matrix between test suites and mutation branches was obtained based on weak mutation criterion.Then,four invalid test requirements and subset test suites were reduced repeatedly.Finally,the current optimal test suite was selected by using artificial fish swarm algorithm,and the simplification and test suite selection operations were performed alternately until all the test requirements were covered.Compared with Greedy algorithm and HGS(Harrold-Gupta-Soff)algorithm on six classical programs,when using weak mutation criterion with no changing or slightly changing mutation score,the reduction rate was improved by 73.4% and 8.2% respectively,and the time consumption was decreased by 25.3% and 56.1% respectively.The experimental results show that the proposed method can effectively reduce the test suites and save the test cost in regression testing.
In view of the problem that the test cost is increased by a large number of test suites in regression testing,a test suite reduction method based on weak mutation criterion was proposed.Firstly,the relation matrix between test suites and mutation branches was obtained based on weak mutation criterion.Then,four invalid test requirements and subset test suites were reduced repeatedly.Finally,the current optimal test suite was selected by using artificial fish swarm algorithm,and the simplification and test suite selection operations were performed alternately until all the test requirements were covered.Compared with Greedy algorithm and HGS(Harrold-Gupta-Soff)algorithm on six classical programs,when using weak mutation criterion with no changing or slightly changing mutation score,the reduction rate was improved by 73.4% and 8.2% respectively,and the time consumption was decreased by 25.3% and 56.1% respectively.The experimental results show that the proposed method can effectively reduce the test suites and save the test cost in regression testing.
引文
[1]NARDO D D,ALSHAHWAN N,BRIAND L,et al.Coveragebased regression test case selection,minimization and prioritization:a case study on an industrial system[J].Software Testing,Verification and Reliability,2015,25(4):371-396.
[2]JONES J A,HARROLD M J.Test-suite reduction and prioritization for modified condition/decision coverage[J].IEEE Transactions on Software Engineering,2003,29(3):195-209.
[3]郑炜,杨喜兵,胡圣佑,等.基于变异分析和覆盖准则的回归测试用例集缩减[J].西北工业大学学报,2017,35(3):494-499.(ZHENG W,YANG X B,HU S Y,et al.Regression test case reduction based on mutation analysis and coverage criterion[J].Journal of Northwestern Polytechnical University,2017,35(3):494-499.)
[4]华丽,李晓月,王成勇,等.能提高错误检测能力的回归测试用例集约简[J].湖南科技大学学报(自然科学版),2015,30(2):99-103.(HUA L,LI X Y,WANG C Y,et al.Regression test suite reduction on improving fault detection capability[J].Journal of Hunan University of Science&Technology(Natural Science Edition),2015,30(2):99-103.)
[5]王曙燕,陈朋媛,孙家泽.基于变异分析的测试用例约简方法[J].计算机应用,2017,37(12):3592-3596.(WANG S Y,CHEN P Y,SUN J Z.Reduction method of test suites based on mutation analysis[J].Journal of Computer Applications,2017,37(12):3592-3596.)
[6]聂长海,徐宝文.一种最小测试用例集生成方法[J].计算机学报,2003,26(12):1690-1695.(NIE C H,XU B W.A minimal test suite generation method[J].Chinese Journal of Computers,2003,26(12):1690-1695.)
[7]HARROLD M J,GUPTA R,SOFFA M L.A methodology for controlling the size of a test suite[J].ACM Transactions on Software Engineering&Methodology,1993,2(3):270-285.
[8]CHEN T Y,LAU M F.A new heuristic for test suite reduction[J].Information and Software Technology,1998,40(5/6):347-354.
[9]LEE J G,CHUNG C G.An optimal representative set selection method[J].Information and Software Technology,2000,42(1):17-25.
[10]MARCHETTO A,SCANNIELLO G,SUSI A.Combining code and requirements coverage with execution cost for test suite reduction[J/OL].IEEE Transactions on Software Engineering,2017[2018-03-06].https://www.onacademic.com/detail/journal_1000040248934010_36b2.html.
[11]VAHABZADEH A,STOCCO A,MESBAH A.Fine-grained test minimization[C]//Proceedings of the 40th International Conference on Software Engineering.New York:ACM,2018:210-221.
[12]章晓芳,徐宝文,聂长海,等.一种基于测试需求约简的测试用例集优化方法[J].软件学报,2007,18(4):821-831.(ZH-NAG X F,XU B W,NIE C H,et al.An approach for optimizing test suite based on testing requirement reduction[J].Journal of Software,2007,18(4):821-831.)
[13]顾庆,唐宝,陈道蓄.一种面向测试需求部分覆盖的测试用例集约简技术[J].计算机学报,2011,34(5):879-888.(GU Q,TANG B,CHEN D X.A test suite reduction technique for partial coverage of test requirements[J].Chinese Journal of Computers,2011,34(5):879-888.)
[14]USAOLA M P,MATEO P R,LAMANCHA B P.Reduction of test suites using mutation[C]//Proceedings of the 2012 International Conference on Fundamental Approaches to Software Engineering,LNCS 7212.Berlin:Springer,2012:425-438.
[15]PAPADAKIS M,MALEVRIS N.Automatically performing weak mutation with the aid of symbolic execution,concolic testing and search-based testing[J].Software Quality Journal,2011,19(4):691-723.
[16]MRESA E S,BOTTACI L.Efficiency of mutation operators and selective mutation strategies:an empirical study[J].Software Testing Verification&Reliability,2015,9(4):205-232.
[17]DEMILLI R A,OFFUTT A J.Constraint-based automatic test data generation[J].IEEE Transactions on Software Engineering,1991,17(9):900-910.
[18]张功杰,巩敦卫,姚香娟.基于统计占优分析的变异测试[J].软件学报,2015,26(10):2504-2520.(ZHANG G J,GONG DW,YAO X J.Mutation testing based on statistical dominance analysis[J].Journal of Software,2015,26(10):2504-2520.)
[19]IIDA C,TAKADA S.Reducing mutants with mutant killable precondition[C]//Proceedings of the 2017 IEEE International Conference on Software Testing,Verification and Validation Workshops.Piscataway,NJ:IEEE,2017:128-133.
[20]李晓磊,邵之江,钱积新.一种基于动物自治体的寻优模式:鱼群算法[J].系统工程理论与实践,2002,22(11):32-38.(LIX L,SHAO Z J,QIAN J X.An optimizing method based on autonomous animats:fish-swarm algorithm[J].Systems Engineering-Theory&Practice,2002,22(11):32-38.)
[21]张功杰,巩敦卫,姚香娟.基于变异分析和集合进化的测试用例生成方法[J].计算机学报,2015,38(11):2318-2331.(ZHANG G J,GONG D W,YAO X J.Test case generation based on mutation analysis and set evolution[J].Chinese Journal of Computers,2015,38(11):2318-2331.).
[2]JONES J A,HARROLD M J.Test-suite reduction and prioritization for modified condition/decision coverage[J].IEEE Transactions on Software Engineering,2003,29(3):195-209.
[3]郑炜,杨喜兵,胡圣佑,等.基于变异分析和覆盖准则的回归测试用例集缩减[J].西北工业大学学报,2017,35(3):494-499.(ZHENG W,YANG X B,HU S Y,et al.Regression test case reduction based on mutation analysis and coverage criterion[J].Journal of Northwestern Polytechnical University,2017,35(3):494-499.)
[4]华丽,李晓月,王成勇,等.能提高错误检测能力的回归测试用例集约简[J].湖南科技大学学报(自然科学版),2015,30(2):99-103.(HUA L,LI X Y,WANG C Y,et al.Regression test suite reduction on improving fault detection capability[J].Journal of Hunan University of Science&Technology(Natural Science Edition),2015,30(2):99-103.)
[5]王曙燕,陈朋媛,孙家泽.基于变异分析的测试用例约简方法[J].计算机应用,2017,37(12):3592-3596.(WANG S Y,CHEN P Y,SUN J Z.Reduction method of test suites based on mutation analysis[J].Journal of Computer Applications,2017,37(12):3592-3596.)
[6]聂长海,徐宝文.一种最小测试用例集生成方法[J].计算机学报,2003,26(12):1690-1695.(NIE C H,XU B W.A minimal test suite generation method[J].Chinese Journal of Computers,2003,26(12):1690-1695.)
[7]HARROLD M J,GUPTA R,SOFFA M L.A methodology for controlling the size of a test suite[J].ACM Transactions on Software Engineering&Methodology,1993,2(3):270-285.
[8]CHEN T Y,LAU M F.A new heuristic for test suite reduction[J].Information and Software Technology,1998,40(5/6):347-354.
[9]LEE J G,CHUNG C G.An optimal representative set selection method[J].Information and Software Technology,2000,42(1):17-25.
[10]MARCHETTO A,SCANNIELLO G,SUSI A.Combining code and requirements coverage with execution cost for test suite reduction[J/OL].IEEE Transactions on Software Engineering,2017[2018-03-06].https://www.onacademic.com/detail/journal_1000040248934010_36b2.html.
[11]VAHABZADEH A,STOCCO A,MESBAH A.Fine-grained test minimization[C]//Proceedings of the 40th International Conference on Software Engineering.New York:ACM,2018:210-221.
[12]章晓芳,徐宝文,聂长海,等.一种基于测试需求约简的测试用例集优化方法[J].软件学报,2007,18(4):821-831.(ZH-NAG X F,XU B W,NIE C H,et al.An approach for optimizing test suite based on testing requirement reduction[J].Journal of Software,2007,18(4):821-831.)
[13]顾庆,唐宝,陈道蓄.一种面向测试需求部分覆盖的测试用例集约简技术[J].计算机学报,2011,34(5):879-888.(GU Q,TANG B,CHEN D X.A test suite reduction technique for partial coverage of test requirements[J].Chinese Journal of Computers,2011,34(5):879-888.)
[14]USAOLA M P,MATEO P R,LAMANCHA B P.Reduction of test suites using mutation[C]//Proceedings of the 2012 International Conference on Fundamental Approaches to Software Engineering,LNCS 7212.Berlin:Springer,2012:425-438.
[15]PAPADAKIS M,MALEVRIS N.Automatically performing weak mutation with the aid of symbolic execution,concolic testing and search-based testing[J].Software Quality Journal,2011,19(4):691-723.
[16]MRESA E S,BOTTACI L.Efficiency of mutation operators and selective mutation strategies:an empirical study[J].Software Testing Verification&Reliability,2015,9(4):205-232.
[17]DEMILLI R A,OFFUTT A J.Constraint-based automatic test data generation[J].IEEE Transactions on Software Engineering,1991,17(9):900-910.
[18]张功杰,巩敦卫,姚香娟.基于统计占优分析的变异测试[J].软件学报,2015,26(10):2504-2520.(ZHANG G J,GONG DW,YAO X J.Mutation testing based on statistical dominance analysis[J].Journal of Software,2015,26(10):2504-2520.)
[19]IIDA C,TAKADA S.Reducing mutants with mutant killable precondition[C]//Proceedings of the 2017 IEEE International Conference on Software Testing,Verification and Validation Workshops.Piscataway,NJ:IEEE,2017:128-133.
[20]李晓磊,邵之江,钱积新.一种基于动物自治体的寻优模式:鱼群算法[J].系统工程理论与实践,2002,22(11):32-38.(LIX L,SHAO Z J,QIAN J X.An optimizing method based on autonomous animats:fish-swarm algorithm[J].Systems Engineering-Theory&Practice,2002,22(11):32-38.)
[21]张功杰,巩敦卫,姚香娟.基于变异分析和集合进化的测试用例生成方法[J].计算机学报,2015,38(11):2318-2331.(ZHANG G J,GONG D W,YAO X J.Test case generation based on mutation analysis and set evolution[J].Chinese Journal of Computers,2015,38(11):2318-2331.).