面向持续集成测试优化的强化学习奖励机制
|
摘要
持续集成环境下的测试存在测试用例集变化大、测试时间有限和快速反馈等需求,传统的测试优化方法难以适用.强化学习是机器学习的一个重要分支,其本质是解决序贯决策问题,可以用于持续集成测试优化.但现有的基于强化学习的方法中,奖励函数计算只包括测试用例在当前集成周期的执行信息.从奖励函数设计和奖励策略两个方面开展研究.在奖励函数设计方面,采用测试用例的完整历史执行信息替代当前执行信息,综合考虑测试用例历史失效总次数和历史失效分布信息,提出了两种奖励函数.在奖励策略方面,提出对当前执行序列的测试用例整体奖励和仅对失效测试用例的部分奖励两种策略.在3个工业级被测程序进行实验研究,结果表明:(1)与现有方法相比,所提出的基于完整历史执行信息奖励函数的强化学习方法可以大幅度提高持续集成测试序列的检错能力;(2)测试用例历史失效分布有助于发现潜在失效的测试用例,对强化学习奖励函数的设计更加重要;(3)整体奖励与部分奖励两种奖励策略受到被测程序的多种因素影响,需要根据实际情况具体选择;(4)包含历史信息的奖励函数会增加时间消耗,但并不影响测试效率.
Testing in continuous integration environment is characterized by constantly changing test sets, limited test time, fast feedback, and so on. Traditional test optimization methods are not suitable for this. Reinforcement learning is an important branch of machine learning, and its essence is to solve sequential decision problems, thus it can be used in test optimization in continuous integration. However, in the existing reinforcement learning based methods, the reward function calculation only includes the execution information of the test case in the current integration cycle. The research is carried out from two aspects: reward function design and reward strategy. In the design of reward function, complete historical execution information of the test case is used to replace the current execution information and the total number of historical failures and historical failure distribution information of the test case is also considered. In terms of the reward strategy, two reward strategies are proposed, which are overall reward for test cases in current execution sequence and partial reward only for failed test cases. In this study, experimental research is conducted on three industrial-level programs. The results show that:(1) Compared with the existing methods, reinforcement learning methods based on reward function with complete historical information proposed in this study can greatly improve the error detection ability of test sequences in continuous integration;(2) Test case historical failure distribution can help to identify potential failure test cases, which is more important for the design of the reward function in reinforcement learning;(3) The two reward strategies, i.e. overall reward and partial reward, are influenced by various factors of the system under test, therefore the reward strategy need to be selected according to the actual situation;and(4) History-based reward functions have longer time consumption, though the test efficiency is not affected.
Testing in continuous integration environment is characterized by constantly changing test sets, limited test time, fast feedback, and so on. Traditional test optimization methods are not suitable for this. Reinforcement learning is an important branch of machine learning, and its essence is to solve sequential decision problems, thus it can be used in test optimization in continuous integration. However, in the existing reinforcement learning based methods, the reward function calculation only includes the execution information of the test case in the current integration cycle. The research is carried out from two aspects: reward function design and reward strategy. In the design of reward function, complete historical execution information of the test case is used to replace the current execution information and the total number of historical failures and historical failure distribution information of the test case is also considered. In terms of the reward strategy, two reward strategies are proposed, which are overall reward for test cases in current execution sequence and partial reward only for failed test cases. In this study, experimental research is conducted on three industrial-level programs. The results show that:(1) Compared with the existing methods, reinforcement learning methods based on reward function with complete historical information proposed in this study can greatly improve the error detection ability of test sequences in continuous integration;(2) Test case historical failure distribution can help to identify potential failure test cases, which is more important for the design of the reward function in reinforcement learning;(3) The two reward strategies, i.e. overall reward and partial reward, are influenced by various factors of the system under test, therefore the reward strategy need to be selected according to the actual situation;and(4) History-based reward functions have longer time consumption, though the test efficiency is not affected.
引文
[1]Busjaeger B,Xie T.Learning for test prioritization:An industrial case study.In:Foundations of Software Engineering.2016.975-980.
[2]Spieker H,Gotlieb A,Marijan D,et al.Reinforcement learning for automatic test case prioritization and selection in continuous integration.In:Proc.of the Int’l Symp.on Software Testing and Analysis.2017.12-22.
[3]Bian Y,Yuan F,Guo JX,Li Z,Zhao RL.CPU+GPU heterogeneous computing orientated multi-objective test case prioritization.Ruan Jian Xue Bao/Journal of Software,2016,27(4):943-954(in Chinese with English abstract).http://www.jos.org.cn/1000-9825/4968.htm[doi:10.13328/j.cnki.jos.004968][3]边毅,袁方,郭俊霞,李征,赵瑞莲.面向CPU+GPU异构计算的多目标测试用例优先排序.软件学报,2016,27(4):943-954.http://www.jos.org.cn/1000-9825/4968.htm[doi:10.13328/j.cnki.jos.004968]
[4]Wong WE,Horgan JR,London S,Agrawal H.A study of effective regression testing in practice.In:Proc.of the Int’l Symp.on Software Reliability Engineering.1997.264-274.
[5]Rothermel G,Untch RH,Chu C,Harrold MJ.Prioritizing test cases for regression testing.IEEE Trans.on Software Engineering,2001,27(10):929-948.
[6]Yoo S,Harman M.Regression testing minimization,selection and prioritization:A survey.Software Testing,Verification&Reliability,2012,22(2):67-120.
[7]Li Z,Harman M,Hierons RM.Search algorithms for regression test case prioritization.IEEE Trans.on Software Engineering,2007,33(4):225-237.
[8]Li Z,Bian Y,Zhao RL,et al.A fine-grained parallel multi-objective test case prioritization on GPU.In:Proc.of the Symp.on Search Based Software Engineering.2013.111-125.
[9]Bian Y,Li Z,Zhao RL,et al.Epistasis based ACO for regression test case prioritization.IEEE Trans.on Emerging Topics in Computational Intelligence,2017,1(3):213-223.
[10]De Souza LS,De Miranda PB,Prudencio RB,Barros FA.A multi-objective particle swarm optimization for test case selection based on functional requirements coverage and execution effort.In:Proc.of the Int’l Conf.on Tools with Artificial Intelligence.2011.245-252.
[11]Yu L,Xu L,Tsai WT.Time-constrained test selection for regression testing.In:Proc.of the Advanced Data Mining and Applications.Springer-Verlag,2010.221-232.
[12]Chen S,Chen Z,Zhao Z,et al.Using semi-supervised clustering to improve regression test selection techniques.In:Proc.of the Int’l Conf.on Software Testing Verification and Validation.2011.1-10.
[13]Groce A,Fern A,Pinto J,Bauer T,Alipour A,Erwig M,Lopez C.Lightweight automated testing with adaptation-based programming.In:Proc.of the Int’l Symp.on Software Reliability Engineering.2012.161-170.
[14]Reichstaller A,Eberhardinger B,Knapp A,Reif W,Gehlen M.Risk-based interoperability testing using reinforcement learning.In:Proc.of the Int’l Conf.on Testing Software and Systems.2016.52-69.
[15]Marijan D,Gotlieb A,Sen S,et al.Test case prioritization for continuous regression testing:An industrial case study.In:Proc.of the Int’l Conf.on Software Maintenance.2013.540-543.
[16]Noor TB,Hemmati H.A similarity-based approach for test case prioritization using historical failure data.In:Proc.of the 2015IEEE 26th Int’l Symp.on Software Reliability Engineering(ISSRE).IEEE,2015.58-68.
[17]Elbaum SG,Rothermel G,Penix J,et al.Techniques for improving regression testing in continuous integration development environments.In:Foundations of Software Engineering.2014.235-245.
[18]Ammar A,Baharom S,Ghani AA,Din J.Enhanced weighted method for test case prioritization in regression testing using unique priority value.In:Proc.of the Int’l Conf.on Information Systems Security.2016.1-6.
[19]Strandberg PE,Sundmark D,Afzal W,Ostrand TJ,Weyuker EJ.Experience report:Automated system level regression test prioritization using multiple factors.In:Proc.of the Int’l Symp.on Software Reliability Engineering.2016.12-23.
[20]Zimmermann T,Premraj R,Zeller A.Predicting defects for eclipse.In:Proc.of the 3rd Int’l Workshop on Predictor Models in Software Engineering(PROMISE 2007).IEEE,2007.
[21]Noor T,Hemmati H.Test case analytics:Mining test case traces to improve risk-driven testing.In:Proc.of the 2015 IEEE 1st Int’l Workshop on Software Analytics(SWAN).IEEE,2015.13-16.
[2]Spieker H,Gotlieb A,Marijan D,et al.Reinforcement learning for automatic test case prioritization and selection in continuous integration.In:Proc.of the Int’l Symp.on Software Testing and Analysis.2017.12-22.
[3]Bian Y,Yuan F,Guo JX,Li Z,Zhao RL.CPU+GPU heterogeneous computing orientated multi-objective test case prioritization.Ruan Jian Xue Bao/Journal of Software,2016,27(4):943-954(in Chinese with English abstract).http://www.jos.org.cn/1000-9825/4968.htm[doi:10.13328/j.cnki.jos.004968][3]边毅,袁方,郭俊霞,李征,赵瑞莲.面向CPU+GPU异构计算的多目标测试用例优先排序.软件学报,2016,27(4):943-954.http://www.jos.org.cn/1000-9825/4968.htm[doi:10.13328/j.cnki.jos.004968]
[4]Wong WE,Horgan JR,London S,Agrawal H.A study of effective regression testing in practice.In:Proc.of the Int’l Symp.on Software Reliability Engineering.1997.264-274.
[5]Rothermel G,Untch RH,Chu C,Harrold MJ.Prioritizing test cases for regression testing.IEEE Trans.on Software Engineering,2001,27(10):929-948.
[6]Yoo S,Harman M.Regression testing minimization,selection and prioritization:A survey.Software Testing,Verification&Reliability,2012,22(2):67-120.
[7]Li Z,Harman M,Hierons RM.Search algorithms for regression test case prioritization.IEEE Trans.on Software Engineering,2007,33(4):225-237.
[8]Li Z,Bian Y,Zhao RL,et al.A fine-grained parallel multi-objective test case prioritization on GPU.In:Proc.of the Symp.on Search Based Software Engineering.2013.111-125.
[9]Bian Y,Li Z,Zhao RL,et al.Epistasis based ACO for regression test case prioritization.IEEE Trans.on Emerging Topics in Computational Intelligence,2017,1(3):213-223.
[10]De Souza LS,De Miranda PB,Prudencio RB,Barros FA.A multi-objective particle swarm optimization for test case selection based on functional requirements coverage and execution effort.In:Proc.of the Int’l Conf.on Tools with Artificial Intelligence.2011.245-252.
[11]Yu L,Xu L,Tsai WT.Time-constrained test selection for regression testing.In:Proc.of the Advanced Data Mining and Applications.Springer-Verlag,2010.221-232.
[12]Chen S,Chen Z,Zhao Z,et al.Using semi-supervised clustering to improve regression test selection techniques.In:Proc.of the Int’l Conf.on Software Testing Verification and Validation.2011.1-10.
[13]Groce A,Fern A,Pinto J,Bauer T,Alipour A,Erwig M,Lopez C.Lightweight automated testing with adaptation-based programming.In:Proc.of the Int’l Symp.on Software Reliability Engineering.2012.161-170.
[14]Reichstaller A,Eberhardinger B,Knapp A,Reif W,Gehlen M.Risk-based interoperability testing using reinforcement learning.In:Proc.of the Int’l Conf.on Testing Software and Systems.2016.52-69.
[15]Marijan D,Gotlieb A,Sen S,et al.Test case prioritization for continuous regression testing:An industrial case study.In:Proc.of the Int’l Conf.on Software Maintenance.2013.540-543.
[16]Noor TB,Hemmati H.A similarity-based approach for test case prioritization using historical failure data.In:Proc.of the 2015IEEE 26th Int’l Symp.on Software Reliability Engineering(ISSRE).IEEE,2015.58-68.
[17]Elbaum SG,Rothermel G,Penix J,et al.Techniques for improving regression testing in continuous integration development environments.In:Foundations of Software Engineering.2014.235-245.
[18]Ammar A,Baharom S,Ghani AA,Din J.Enhanced weighted method for test case prioritization in regression testing using unique priority value.In:Proc.of the Int’l Conf.on Information Systems Security.2016.1-6.
[19]Strandberg PE,Sundmark D,Afzal W,Ostrand TJ,Weyuker EJ.Experience report:Automated system level regression test prioritization using multiple factors.In:Proc.of the Int’l Symp.on Software Reliability Engineering.2016.12-23.
[20]Zimmermann T,Premraj R,Zeller A.Predicting defects for eclipse.In:Proc.of the 3rd Int’l Workshop on Predictor Models in Software Engineering(PROMISE 2007).IEEE,2007.
[21]Noor T,Hemmati H.Test case analytics:Mining test case traces to improve risk-driven testing.In:Proc.of the 2015 IEEE 1st Int’l Workshop on Software Analytics(SWAN).IEEE,2015.13-16.