基于消息机制的软件自动测试技术研究
|
摘要
软件测试方法研究是软件工程技术中的重要研究分支之一。统计表明，在典型的软件开发项目中，软件测试的工作量占到了软件开发总工作量的40％甚至更多。然而由于软件测试中的许多工作非常繁琐和耗时，用传统的手工方式难以完成，必须以自动测试手段实现。
如何对人机交互频繁的软件进行自动测试是一个富有挑战性的研究课题。已有的自动测试理论和方法如界面类对象建模技术、基于Z规格的测试用例自动生成技术、基于SOFL规约的测试用例生成模型虽然在一定程度上提高了功能测试的形式化程度，但在针对如三维数字化设计系统这类人机交互复杂、系统状态繁多的软件系统时，因复杂度高而实现难度大。
本课题研究以国家863基础类研究课题“三维数字化设计系统自动测试技术研究与系统实现(课题编号：2002AA411510)”为背景，基于Windows消息机制提出一种针对复杂图形用户界面软件进行自动化黑盒测试的构造模型，该模型通过拦截Windows底层的各种消息，同时依据相应的流程规约和生成算法将原始消息序列进行重组，从而派生出大量的测试用例，然后以重构后的衍生消息序列驱动被测试系统自动运行，达到自动化黑盒测试的目的。
本文最后依此测试模开构建出一个基于消息机制的自动测试系统CADTesting。实验结果表明，基于该思路构建的测试系统具有良好的测试效果，能够较大程度上减轻测试人员劳动强度、降低测试成本并提高测试效率，该系统可在回归测试(Regression Test)、压力测试(Stress Test)、性能测试(Performance Test)中推广应用。0
Software Testing, as an important branch of Software Engineering, is a new and vital subject of research. Statistics show that, in typical software development projects, the work of software testing accounts for 40% or more of all. Due to the monotony, complexity, and time-consumingness of most work in software testing, it is hard to accomplish by traditional manual work, therefore, automated testing approaches should be employed.
How to make an automated Black-box testing on human-machine interactive software is a challenging subject. The existing theories and methods of automated testing, like modeling technology of interface objects, Z-specification-based testing sample auto-generating technology, and SOFL-based testing sample generating model, improved, to a certain extent, the formalization level function testing. However, in regard to complicatedly human-machine interactive software systems of various systematic states, such as CAD system, these methods are hard to function.
Our research is supported by a national 863 research project which is named as Research on Automatic Test Technology for 3-Dimensions CAD System (Project No. is 2002AA411510). This thesis, basing on Windows Message Mechanism, proposes a structural model of automated black box testing aimed at graphic user interface software. Through picking up various message at the bottom of Windows and meanwhile reorganizing the message order according to the relevant flow specification and generative calculation, this model derives abundant testing samples and then drives the system being tested to work automatically by the reorganized derivative message order, thus achieving the goal of automated testing.
According to this, a message-mechanism based automated testing system is constructed in this thesis. Experiments show that the testing system constructed following this idea, by assuaging the work intensity of testing staff, lower the testing
cost, and improve the efficiency of testing, achieves good testing results. This system can be widely applied in regression testing.
如何对人机交互频繁的软件进行自动测试是一个富有挑战性的研究课题。已有的自动测试理论和方法如界面类对象建模技术、基于Z规格的测试用例自动生成技术、基于SOFL规约的测试用例生成模型虽然在一定程度上提高了功能测试的形式化程度，但在针对如三维数字化设计系统这类人机交互复杂、系统状态繁多的软件系统时，因复杂度高而实现难度大。
本课题研究以国家863基础类研究课题“三维数字化设计系统自动测试技术研究与系统实现(课题编号：2002AA411510)”为背景，基于Windows消息机制提出一种针对复杂图形用户界面软件进行自动化黑盒测试的构造模型，该模型通过拦截Windows底层的各种消息，同时依据相应的流程规约和生成算法将原始消息序列进行重组，从而派生出大量的测试用例，然后以重构后的衍生消息序列驱动被测试系统自动运行，达到自动化黑盒测试的目的。
本文最后依此测试模开构建出一个基于消息机制的自动测试系统CADTesting。实验结果表明，基于该思路构建的测试系统具有良好的测试效果，能够较大程度上减轻测试人员劳动强度、降低测试成本并提高测试效率，该系统可在回归测试(Regression Test)、压力测试(Stress Test)、性能测试(Performance Test)中推广应用。0
Software Testing, as an important branch of Software Engineering, is a new and vital subject of research. Statistics show that, in typical software development projects, the work of software testing accounts for 40% or more of all. Due to the monotony, complexity, and time-consumingness of most work in software testing, it is hard to accomplish by traditional manual work, therefore, automated testing approaches should be employed.
How to make an automated Black-box testing on human-machine interactive software is a challenging subject. The existing theories and methods of automated testing, like modeling technology of interface objects, Z-specification-based testing sample auto-generating technology, and SOFL-based testing sample generating model, improved, to a certain extent, the formalization level function testing. However, in regard to complicatedly human-machine interactive software systems of various systematic states, such as CAD system, these methods are hard to function.
Our research is supported by a national 863 research project which is named as Research on Automatic Test Technology for 3-Dimensions CAD System (Project No. is 2002AA411510). This thesis, basing on Windows Message Mechanism, proposes a structural model of automated black box testing aimed at graphic user interface software. Through picking up various message at the bottom of Windows and meanwhile reorganizing the message order according to the relevant flow specification and generative calculation, this model derives abundant testing samples and then drives the system being tested to work automatically by the reorganized derivative message order, thus achieving the goal of automated testing.
According to this, a message-mechanism based automated testing system is constructed in this thesis. Experiments show that the testing system constructed following this idea, by assuaging the work intensity of testing staff, lower the testing
cost, and improve the efficiency of testing, achieves good testing results. This system can be widely applied in regression testing.
引文
[1] Cem Kaner，James Bach，Bret Pettichord．软件测试经验与教训．北京：机械工业出版社，2003
[2] 张克东，庄燕滨．软件工程与软件测试自动化教程．北京：电子工业出版社，2002
[3] Myers GJ. The Art of Software Testing. New York: John Wiley & Sons Inc., 1979
[4] Paul C．Jorgensen．软件测试．北京：机械工业出版社，2003
[5] A Jefferson 0ffutt, Shaoying Liu. Generating testing data from SOFL specification. The Journal of Systems and Software, 1999,49(1):49-62
[6] John Anil Saldhana, Sol M Shatz, ZhaoXia Hu. Formalization of object behavior and interactions from UML models. International Journal of Software Engineering Knowledge Engineering, 2001, 11 (6): 643-673
[7] B.A. Myers. User Interface Software Tools, ACM Trans. Computer-Human Interaction, 1995, 2(1):64-103
[8] R.K. Shehady, D.P. Siewiorek. A Method to Automate User Interface Testing Using Variable Finite State Machines, Proc. 27th Ann. Int'l Symp. Fault-Tolerant Computing (FTCS'97), 1997, 80-88
[9] Parrish A S, BorieR B, Cordes D W. Automated flow graph-based testing of object-oriented software modules, J Systems Software, 1993, 23:95-109
[10] David A.Wooff, Michael Goldstein, Frank RA.Coolen. Bayesian Graphical Models for Software Testing.. IEEE Trans on Software Engineering, 2002,5(28):510-525
[11] A.M. Memon, M.E. Pollack, and M.L. Soffa. Using a Goal-Driven Approach to Generate Test Cases for GUIs, Proc. 21st Int'l Conf. Software Eng., 1999, 3:257-266
[12] Atif M. Memon. Hierarchical GUI test case generation using automated planning. IEEETransactions on Software Engineering, 2001,27(2):144-155
[13] Atif M. Memon, Mary Lou Soffa. Regression testing of GUls. ESEC/FSE'03 ACM, 2003,5(1):118-127
[14] Atif M. Memon. GUI Testing: Pitfalls and process. IEEE Computer, 2002,35(8):90-91
[15] Atif. M. Memon, M. E. Pollack, M. L. Soffa. Automated test oracles for GUIs. In Proceedings of the ACM, SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-8), 2000,30-39
[16] Gregg Rothermel, Mary Jean Harrold. Empirical Studies of a Safe Regression Test Selection Technique[J].IEEE trans on Software Engineering, 1998,6
[17] Sebastian Elbaum. Test Case Prioritization:A Family of Empirical Studies[J].IEEE Trans on Software Engineering, 2002,6
[18] Gregg Rothermel, Mary Jean Harrold. Analyzing Regression Test Selection Techniques[J].IEEE trans on Software Engineering, 1996,8
[19] 顾玉良，王立福．界面类对象建模技术研究．计算机工程，1999，25(7)：21-23
[20] 刘超，程序交互执行流程图及其测试覆盖准则．软件学报，1998，9(6)：458-463
[21] 杜栓柱．基于界面构件关联图的软件功能测试技术．计算机研究与发展，2002，39(2)：148～152
[22] 兰毓华，毛法尧，曹化工．基于Z规格说明的软件测试用例自动生成．计算机学报，1999，22(9)：963～969
[23] 何允如，刘宗田，郝峰等．基于程序执行状态图的软件测试方法研究．计算机工程与应用，2002，18：96～98
[24] 李留英，王戟，齐治昌．UML Statechart 图的操作语义．软件学报，2001，12(12)：1864—1873
[25] 李留英，王戟，齐治昌．UML Statecharts的测试用例生成方法．计算机研究与发展，2001，38(6)：691-697
[26] Liuying Li, ZhiChang Qi. Test selection from UML Statecharts. in: 31st International Conference on Technology of Object-Oriented Language and Systems. Washington, DC: IEEE CS Press, 1999, 273-279
[27] 单锦辉，齐治昌，吴建平．Gupta方法的改进．计算机学报，2002，25(12)：1378—1386
[28] Rex Black．软件测试过程管理．北京：机械工业出版社，2003
[29] Johnson M．Hart．win32系统编程—Windows 2000应用程序开发指南．北京：中国电力出版社，2003
[30] David A. Solomon. Windows 2000技术内幕．北京：北京大学出版社，2001
[31] Microsoft Crop. MFC与Windows编程．北京：北京大学出版社，2000
[32] Jeffrey Richter. Windows高级编程指南．北京：清华大学出版社，1999
[33] David CKung, Jerry Gao, Pei Hsia. On regression testing of object-oriented programs. Journal of Systems Software, 1996,32(1):21～40
[34] Zhu H A formal analysis of the subsume relation between software test adequacy criteria. IEEE Transactions on Software Engineering, 1996, 22(4):248-255
[35] Mark Fewster, Dorothy Graham.软件测试自动化技术与实例详解．北京：电子：工业出版社，2000，85—122
[36] John W. Satzinger Rober B. Jackson Stephen D. Burd.系统分析与设计．北京：机械工业出版社，2002，287-337
[37] 戴春达，符红光．Win32中钩子的实现技术及其应用．计算机应用．2002，8：72—74
[38] D. Rosenberg. User Interface Prototyping Paradigms in the 90' s, Proc. Conf. Human Factors in Computing Systems-Adjunct Proc. (ACM INTERCHI '93), 1993, 231-238
[39] M.G. El-Said, G. Fischer, S.A. Gamalel-Din, and M. Zaki. ADDI: A Tool for Automating the Design of Visual Interfaces, Computers & Graphics, 1997,21(1):79-87
[40] A. Howe, A. von Mayrhauser, and R.T. Mraz. Test Case Generation as an AI Planning Problem. Automated Software Eng., 1997,4:77-106
[41] 蔡开元，李永超，景涛．软件测试的控制论方法．航空学报，2002，23(5)：448-454
[42] 宫云战，高文玲．软件测试与软件可靠性之间的关系．同济大学学报，2002，30(12)：1529—1531
[43] 吴际，金茂忠，刘超，软件复杂性与测试用例集价值．电子学报，2002，30(12)：2166—2168
[44] 曹明，孙昌爱，刘超．一种有效的软件测试工程化方法．计算机工程与应用，2003．5：127-128
[45] 曹化工，刘澎，秦友淑．一种用于软件测试的程序结构划分方法．小型微型计算机系统，2002，23(4)：492-495，
[46] 赵瑞莲，闵应骅．基于谓词切片的字符串测试数据自动生成．计算机研究与发展，2002，39(4)：473—481
[47] 景涛，白成刚，胡庆培．构件软件的测试问题综述．计算机工程与应用，2002，24：1-6
[48] William E．Perry，Randall W．Rice．软件测试求生法则．北京：清华大学出版社，2003
[49] 马毅，李建华．FREE模型在面向对象测试中的应用研究．计算机应用研究，2002．9：88—90
[50] 张雪萍，庄雷，范艳峰．基于状态的类测试技术研究．小型微型计算机系统，2002，9：1121—1124
[51] 顾庆，陈道蓄，谢立．基于有限状态进程的事件约束定义．软件学报，2002，13(11)：2162—2168
[52] 郑亚玲，胡和平．回归测试策略的新领域．计算机应用研究，2000，6：23—26
[53] 暴建民，王申科．关于可应用测试数据充分性准则复杂性的注记．1999，22(4)：437-441
[2] 张克东，庄燕滨．软件工程与软件测试自动化教程．北京：电子工业出版社，2002
[3] Myers GJ. The Art of Software Testing. New York: John Wiley & Sons Inc., 1979
[4] Paul C．Jorgensen．软件测试．北京：机械工业出版社，2003
[5] A Jefferson 0ffutt, Shaoying Liu. Generating testing data from SOFL specification. The Journal of Systems and Software, 1999,49(1):49-62
[6] John Anil Saldhana, Sol M Shatz, ZhaoXia Hu. Formalization of object behavior and interactions from UML models. International Journal of Software Engineering Knowledge Engineering, 2001, 11 (6): 643-673
[7] B.A. Myers. User Interface Software Tools, ACM Trans. Computer-Human Interaction, 1995, 2(1):64-103
[8] R.K. Shehady, D.P. Siewiorek. A Method to Automate User Interface Testing Using Variable Finite State Machines, Proc. 27th Ann. Int'l Symp. Fault-Tolerant Computing (FTCS'97), 1997, 80-88
[9] Parrish A S, BorieR B, Cordes D W. Automated flow graph-based testing of object-oriented software modules, J Systems Software, 1993, 23:95-109
[10] David A.Wooff, Michael Goldstein, Frank RA.Coolen. Bayesian Graphical Models for Software Testing.. IEEE Trans on Software Engineering, 2002,5(28):510-525
[11] A.M. Memon, M.E. Pollack, and M.L. Soffa. Using a Goal-Driven Approach to Generate Test Cases for GUIs, Proc. 21st Int'l Conf. Software Eng., 1999, 3:257-266
[12] Atif M. Memon. Hierarchical GUI test case generation using automated planning. IEEETransactions on Software Engineering, 2001,27(2):144-155
[13] Atif M. Memon, Mary Lou Soffa. Regression testing of GUls. ESEC/FSE'03 ACM, 2003,5(1):118-127
[14] Atif M. Memon. GUI Testing: Pitfalls and process. IEEE Computer, 2002,35(8):90-91
[15] Atif. M. Memon, M. E. Pollack, M. L. Soffa. Automated test oracles for GUIs. In Proceedings of the ACM, SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-8), 2000,30-39
[16] Gregg Rothermel, Mary Jean Harrold. Empirical Studies of a Safe Regression Test Selection Technique[J].IEEE trans on Software Engineering, 1998,6
[17] Sebastian Elbaum. Test Case Prioritization:A Family of Empirical Studies[J].IEEE Trans on Software Engineering, 2002,6
[18] Gregg Rothermel, Mary Jean Harrold. Analyzing Regression Test Selection Techniques[J].IEEE trans on Software Engineering, 1996,8
[19] 顾玉良，王立福．界面类对象建模技术研究．计算机工程，1999，25(7)：21-23
[20] 刘超，程序交互执行流程图及其测试覆盖准则．软件学报，1998，9(6)：458-463
[21] 杜栓柱．基于界面构件关联图的软件功能测试技术．计算机研究与发展，2002，39(2)：148～152
[22] 兰毓华，毛法尧，曹化工．基于Z规格说明的软件测试用例自动生成．计算机学报，1999，22(9)：963～969
[23] 何允如，刘宗田，郝峰等．基于程序执行状态图的软件测试方法研究．计算机工程与应用，2002，18：96～98
[24] 李留英，王戟，齐治昌．UML Statechart 图的操作语义．软件学报，2001，12(12)：1864—1873
[25] 李留英，王戟，齐治昌．UML Statecharts的测试用例生成方法．计算机研究与发展，2001，38(6)：691-697
[26] Liuying Li, ZhiChang Qi. Test selection from UML Statecharts. in: 31st International Conference on Technology of Object-Oriented Language and Systems. Washington, DC: IEEE CS Press, 1999, 273-279
[27] 单锦辉，齐治昌，吴建平．Gupta方法的改进．计算机学报，2002，25(12)：1378—1386
[28] Rex Black．软件测试过程管理．北京：机械工业出版社，2003
[29] Johnson M．Hart．win32系统编程—Windows 2000应用程序开发指南．北京：中国电力出版社，2003
[30] David A. Solomon. Windows 2000技术内幕．北京：北京大学出版社，2001
[31] Microsoft Crop. MFC与Windows编程．北京：北京大学出版社，2000
[32] Jeffrey Richter. Windows高级编程指南．北京：清华大学出版社，1999
[33] David CKung, Jerry Gao, Pei Hsia. On regression testing of object-oriented programs. Journal of Systems Software, 1996,32(1):21～40
[34] Zhu H A formal analysis of the subsume relation between software test adequacy criteria. IEEE Transactions on Software Engineering, 1996, 22(4):248-255
[35] Mark Fewster, Dorothy Graham.软件测试自动化技术与实例详解．北京：电子：工业出版社，2000，85—122
[36] John W. Satzinger Rober B. Jackson Stephen D. Burd.系统分析与设计．北京：机械工业出版社，2002，287-337
[37] 戴春达，符红光．Win32中钩子的实现技术及其应用．计算机应用．2002，8：72—74
[38] D. Rosenberg. User Interface Prototyping Paradigms in the 90' s, Proc. Conf. Human Factors in Computing Systems-Adjunct Proc. (ACM INTERCHI '93), 1993, 231-238
[39] M.G. El-Said, G. Fischer, S.A. Gamalel-Din, and M. Zaki. ADDI: A Tool for Automating the Design of Visual Interfaces, Computers & Graphics, 1997,21(1):79-87
[40] A. Howe, A. von Mayrhauser, and R.T. Mraz. Test Case Generation as an AI Planning Problem. Automated Software Eng., 1997,4:77-106
[41] 蔡开元，李永超，景涛．软件测试的控制论方法．航空学报，2002，23(5)：448-454
[42] 宫云战，高文玲．软件测试与软件可靠性之间的关系．同济大学学报，2002，30(12)：1529—1531
[43] 吴际，金茂忠，刘超，软件复杂性与测试用例集价值．电子学报，2002，30(12)：2166—2168
[44] 曹明，孙昌爱，刘超．一种有效的软件测试工程化方法．计算机工程与应用，2003．5：127-128
[45] 曹化工，刘澎，秦友淑．一种用于软件测试的程序结构划分方法．小型微型计算机系统，2002，23(4)：492-495，
[46] 赵瑞莲，闵应骅．基于谓词切片的字符串测试数据自动生成．计算机研究与发展，2002，39(4)：473—481
[47] 景涛，白成刚，胡庆培．构件软件的测试问题综述．计算机工程与应用，2002，24：1-6
[48] William E．Perry，Randall W．Rice．软件测试求生法则．北京：清华大学出版社，2003
[49] 马毅，李建华．FREE模型在面向对象测试中的应用研究．计算机应用研究，2002．9：88—90
[50] 张雪萍，庄雷，范艳峰．基于状态的类测试技术研究．小型微型计算机系统，2002，9：1121—1124
[51] 顾庆，陈道蓄，谢立．基于有限状态进程的事件约束定义．软件学报，2002，13(11)：2162—2168
[52] 郑亚玲，胡和平．回归测试策略的新领域．计算机应用研究，2000，6：23—26
[53] 暴建民，王申科．关于可应用测试数据充分性准则复杂性的注记．1999，22(4)：437-441