脚本自动生成在面向对象软件测试自动化中的应用
摘要
本文主要研究了在面向对象软件测试自动化中脚本自动生成技术的应用,以及对其过程的改进方法。基于对目前测试方法存在问题的总结和测试工具的局限性,提出了一套比较完备的自动化测试框架,并实现了一个自动化的软件测试工具AutoTest,它包括两个子工具。
对不含GUI软件的测试子工具AutoNonGUITest重点讲述了:自动搜索程序集合测试信息;自动生成和编写测试数据;测试脚本自动生成;在集成测试中重用底层测试脚本构造桩模块;调用测试脚本、自动验证、生成测试结果报告。
对GUI软件的测试子工具AutoGUITest重点讲述了:自动搜索GUI控件;测试用例生成技术;数据驱动通用脚本;测试自动执行方法;自动生成测试验证结果。
接着使用了实例对方法及工具进行验证。经过实践证明,AutoTest测试工具能较方便地实现对面向对象程序的自动化测试。
As indispensable part of software life cycle, software testing plays animportant role in improving the quality of software. With the development ofsoftware technology, the use of software test automation technology to raisetest efficiency, shorten the development cycle and reduce the cost of softwaretesting has become the inevitable trend of development. A lot of work hasbeen done in software testing automation. Many software companies developcommercial software testing tools for automated testing.
The rapid development of object-oriented software technology makesthe research and development of object-oriented software testing tools hasbecome an urgent task. Therefore, this paper will discuss features andmethods of object-oriented software testing, featuring in the application ofautomatic generation of test scripts in object-oriented software testing andtest process improvement.
Graphical user interface GUI (Graphical User Interface) is an importantcomponent of modern software part. Since the main function of the softwareis called through a graphical user interface, modern GUI test is the key tosoftware testing. As the most commonly used interactive media betweencomputer and users, GUI has developed rapidly. With the increasing of typeand function, GUI software not only brought convenience to the users, it hasalso brought challenge to software quality assurance. GUI functional testingmainly tests whether the software achieve users’expectations withoutexception reports. The complexity of the GUI testing determines itsdevelopment direction is how to achieve the greatest degree of automation toimprove efficiency and accuracy.
During testing process, test scripts are usually used to reduce the manualworkload and improve the automation of testing after designing the test cases. Test scripts play an important role in the course of automated testing. Thedesign and creation of scripts directly impact on the test results reproduction.The creation of automated test scripts will greatly improve the automation oftesting.
In this paper, the software testing technology and the status quo ofresearch are firstly introduced. Then the concept, technology and meaning ofautomated testing software, the principle and methods of test automationtools, and the development trend of test automation are introduced. Then, theadvantages and limitations of automated testing are analyzed. Also a detailedanalysis and comparison of current technologies of automatic generation ofthe test scripts is given.
The features and the technology of commonly used automated GUItesting tools are analyzed. Most GUI test tool has not reached the maximumdegree of automation: the capture/playback technology needs manual testdata collection; data-driven test scripts are not really used; no results reportsverification mechanism. At the same time, the use of third-party scriptinglanguage in these tools has brought inconvenience.
Based on the analysis of the problems and the limitations of currenttesting method and software testing tools, after reading a lot of related papers,learning from practical experience, a more complete set of automated testingsolutions which implements a greater degree of automated testing frameworkis put forward. Based on this framework and using .NET technology, anautomated software testing tool AutoTest is implemented.It includesAutoNonGUITest for non-GUI program testing and AutoGUITest for GUItesting. Although AutoTest currently only supports the applications in .NETFramework object-oriented languages, but its ideas and technology can befurther extended to testing under other platforms.
In this paper, functional design and the realization of functional modulesof AutoNonGUITest, which is used to achieve the automation of non-GUI software testing, are illustrated. AutoNonGUITest tools use the smallestinteraction to thoroughly test a complex software product. AutoNonGUITestmakes the testing of non-GUI program more thoroughly and automatically,reducing the workload of test engineer.The following technical principles andrealization processes are discussed:
1) Research on testing principle of data-driven methods of softwaretesting automation which AutoNonGUITest uses.
2) Implementation of automatic search the namespace and class in theprogram, gathering relevant test information.
3) Automatic test data generation and modification. Use XMLdocuments and EXCEL spreadsheet related namespace to implement testingdata access.
4) Implementation of automatic generating mechanism of data-driventest scripts, the separation of the test scripts and the test data, thus enhancingthe rate of reusable test scripts.
5) Simplify the stub modeling of integration testing through the reusingtest scripts of the bottom. Use real objects in the integration testing.
6) Automatic running of test scripts, automatic test validation, automaticgeneration of the tests reports.
AutoGUITest tool achieves a greater degree of GUI testing automation,fixing existing problems of current automated GUI testing tools.
AutoGUITest tool reduces the number of interaction between tester andprogram, simplifies the process of generating and maintaining test cases,avoids tedious manual test validation process.The paper focuses on thefollowing principles and the technical implementation process:
1) Using automatic search of GUI Controls to replace the captureprocess of commercial testing tools. First, automatic searching approach ofGUI needs to divide GUI screen of application under test into a grid system.the size of Each grid is determined by the density of controls under test so that each GUI control must appear in at least one grid.
2) Try to use improved test case generation technology. The technologyfills the test sequence editor with the information of step 1. Tester will specifycontrol and its event from the test data collector, and insert test points (objectattributes to be verified). The system will automatically capture theinformation and generate test cases. Tester will be able to generate test casesquickly and accurately using a little interaction with program.
3) Data-driven universal script: current scene-based test scripts notonly include test case, but also the corresponding scene information anddetails of achieving the bottom. The script has poor readability and difficultyto maintain. The research of this paper is to allow universal test script onlycontains the information of implementation, reducing the underlying sceneinformation, the specific details of the realization and the test data, so thatenhance the readability and maintainability of script greatly.
4) Methods of testing implementation: The testing process is driven bythe test cases. The objective of this paper is the analyzing of test scripts andunderlying realization of the basic operation of the objects in the test cases.
5) Automatic generation of test verification results. Four typical GUIverification methods are discussed: simple object verification, the specificobject verification, sum up validation and optimization certification.
In this paper, some examples are used to verify the characteristics ofAutoGUITest and AutoNonGUITest. These examples prove that the AutoTesttesting tools are able to achieve more convenient object-oriented automatedtest process. Finally, the main work is concluded and the next work of thefuture is prospected.
对不含GUI软件的测试子工具AutoNonGUITest重点讲述了:自动搜索程序集合测试信息;自动生成和编写测试数据;测试脚本自动生成;在集成测试中重用底层测试脚本构造桩模块;调用测试脚本、自动验证、生成测试结果报告。
对GUI软件的测试子工具AutoGUITest重点讲述了:自动搜索GUI控件;测试用例生成技术;数据驱动通用脚本;测试自动执行方法;自动生成测试验证结果。
接着使用了实例对方法及工具进行验证。经过实践证明,AutoTest测试工具能较方便地实现对面向对象程序的自动化测试。
As indispensable part of software life cycle, software testing plays animportant role in improving the quality of software. With the development ofsoftware technology, the use of software test automation technology to raisetest efficiency, shorten the development cycle and reduce the cost of softwaretesting has become the inevitable trend of development. A lot of work hasbeen done in software testing automation. Many software companies developcommercial software testing tools for automated testing.
The rapid development of object-oriented software technology makesthe research and development of object-oriented software testing tools hasbecome an urgent task. Therefore, this paper will discuss features andmethods of object-oriented software testing, featuring in the application ofautomatic generation of test scripts in object-oriented software testing andtest process improvement.
Graphical user interface GUI (Graphical User Interface) is an importantcomponent of modern software part. Since the main function of the softwareis called through a graphical user interface, modern GUI test is the key tosoftware testing. As the most commonly used interactive media betweencomputer and users, GUI has developed rapidly. With the increasing of typeand function, GUI software not only brought convenience to the users, it hasalso brought challenge to software quality assurance. GUI functional testingmainly tests whether the software achieve users’expectations withoutexception reports. The complexity of the GUI testing determines itsdevelopment direction is how to achieve the greatest degree of automation toimprove efficiency and accuracy.
During testing process, test scripts are usually used to reduce the manualworkload and improve the automation of testing after designing the test cases. Test scripts play an important role in the course of automated testing. Thedesign and creation of scripts directly impact on the test results reproduction.The creation of automated test scripts will greatly improve the automation oftesting.
In this paper, the software testing technology and the status quo ofresearch are firstly introduced. Then the concept, technology and meaning ofautomated testing software, the principle and methods of test automationtools, and the development trend of test automation are introduced. Then, theadvantages and limitations of automated testing are analyzed. Also a detailedanalysis and comparison of current technologies of automatic generation ofthe test scripts is given.
The features and the technology of commonly used automated GUItesting tools are analyzed. Most GUI test tool has not reached the maximumdegree of automation: the capture/playback technology needs manual testdata collection; data-driven test scripts are not really used; no results reportsverification mechanism. At the same time, the use of third-party scriptinglanguage in these tools has brought inconvenience.
Based on the analysis of the problems and the limitations of currenttesting method and software testing tools, after reading a lot of related papers,learning from practical experience, a more complete set of automated testingsolutions which implements a greater degree of automated testing frameworkis put forward. Based on this framework and using .NET technology, anautomated software testing tool AutoTest is implemented.It includesAutoNonGUITest for non-GUI program testing and AutoGUITest for GUItesting. Although AutoTest currently only supports the applications in .NETFramework object-oriented languages, but its ideas and technology can befurther extended to testing under other platforms.
In this paper, functional design and the realization of functional modulesof AutoNonGUITest, which is used to achieve the automation of non-GUI software testing, are illustrated. AutoNonGUITest tools use the smallestinteraction to thoroughly test a complex software product. AutoNonGUITestmakes the testing of non-GUI program more thoroughly and automatically,reducing the workload of test engineer.The following technical principles andrealization processes are discussed:
1) Research on testing principle of data-driven methods of softwaretesting automation which AutoNonGUITest uses.
2) Implementation of automatic search the namespace and class in theprogram, gathering relevant test information.
3) Automatic test data generation and modification. Use XMLdocuments and EXCEL spreadsheet related namespace to implement testingdata access.
4) Implementation of automatic generating mechanism of data-driventest scripts, the separation of the test scripts and the test data, thus enhancingthe rate of reusable test scripts.
5) Simplify the stub modeling of integration testing through the reusingtest scripts of the bottom. Use real objects in the integration testing.
6) Automatic running of test scripts, automatic test validation, automaticgeneration of the tests reports.
AutoGUITest tool achieves a greater degree of GUI testing automation,fixing existing problems of current automated GUI testing tools.
AutoGUITest tool reduces the number of interaction between tester andprogram, simplifies the process of generating and maintaining test cases,avoids tedious manual test validation process.The paper focuses on thefollowing principles and the technical implementation process:
1) Using automatic search of GUI Controls to replace the captureprocess of commercial testing tools. First, automatic searching approach ofGUI needs to divide GUI screen of application under test into a grid system.the size of Each grid is determined by the density of controls under test so that each GUI control must appear in at least one grid.
2) Try to use improved test case generation technology. The technologyfills the test sequence editor with the information of step 1. Tester will specifycontrol and its event from the test data collector, and insert test points (objectattributes to be verified). The system will automatically capture theinformation and generate test cases. Tester will be able to generate test casesquickly and accurately using a little interaction with program.
3) Data-driven universal script: current scene-based test scripts notonly include test case, but also the corresponding scene information anddetails of achieving the bottom. The script has poor readability and difficultyto maintain. The research of this paper is to allow universal test script onlycontains the information of implementation, reducing the underlying sceneinformation, the specific details of the realization and the test data, so thatenhance the readability and maintainability of script greatly.
4) Methods of testing implementation: The testing process is driven bythe test cases. The objective of this paper is the analyzing of test scripts andunderlying realization of the basic operation of the objects in the test cases.
5) Automatic generation of test verification results. Four typical GUIverification methods are discussed: simple object verification, the specificobject verification, sum up validation and optimization certification.
In this paper, some examples are used to verify the characteristics ofAutoGUITest and AutoNonGUITest. These examples prove that the AutoTesttesting tools are able to achieve more convenient object-oriented automatedtest process. Finally, the main work is concluded and the next work of thefuture is prospected.
引文
[1] 单锦辉, 姜瑛, 孙萍. 软件测试研究进展. 北京大学学报(自然科学版)2005年01期.
[2] Ron Patton.周予滨,姚静译.软件测试.北京.机械工业出版社.2002.23-25.
[3] B. A. Myers. User interface software tools. ACM Transactions onComputer-HumanInteraction2(1):64-1031995.
[4] Glenford J. Myers. The Art of Software Testing Second Edition. JohnWileyandSons.2004.
[5] LauraRose.GettingtheMostfromYourAutomatedTesting2001.
[6] AtifM.Memon.RegressionTestingofGUIs.ESEC/FSE'032003.
[7] Kanglin Li,Mengqi Wu. Effective GUI Test Automation Developing anAutomatedGUITestingTool.Sybex.2004.
[8] Steve McConnell. 10 Keys to Successful Software Projects AnExebuttonliveGuide.Tech.Presentation.2001.
[9] Boris Beizer. Software Testing Techniques. Van Nostrand reinholdPublishing.1983.
[10]古乐, 史林九著. 软件测试技术概论.北京:清华大学出版社.2004.
[11]惠特克J A著. 马良荔,俞立军译. 实用软件测试指南. 北京:电子工业出版社.2003.
[12]Atif M.Memon. Hierarchical GUI Test Case Generation UsingAutomationPlanning. IEEETransactions onSoftwareEngineering. 2001.27(2).
[13]Atif Memon, Ishan Banerjee,Adithya Nagarajan.GUI Ripping. ReverseEngineeringofGraphicalUserInterfacesforTesting.WCRE’03.2003.
[14]Juichi Takahashi, Yoshiaki Kakuda.Effective Automated Testing. ASolutionofGraphicalObjectVerification.ATS’02.2002.
[15]Atif M. Memon. A comprehensive framework for testing graphical userinterfacesUMIMicroform.2001.
[16]S. Rosaria, H. Robinson. Applying models in your testing process.InformationandSoftwareTechnology42(2000)815-824.
[17]MichaelKelly.Choosingatestautomationframework.2003.
[18]Mark Fewster, Dorothy Graham 著.舒智勇等译.软件测试自动化技术与实例详解.北京:电子工业出版社.2000.
[19]Tassey, Gregory. The Economic Impacts of Inadequate Infrastructure forSoftwareTesting.Collingdale,PA:DIANEPublishingCo,2003.
[20]Dustin, Elfriede. Effective Software Testing: 50 Specific Ways toImproveYourTesting.Boston,Mass:Addison-Wesley,2002.
[21](美)Charles F.Goldfarb, Paul Prescod著.XML实用技术.清华大学出版社,1999.9.
[22]Kanglin Li, Mengqi Wu. Effective Software Test Automation.Sybex.2004.
[23]Atif M. Memon. GUI testing: Pitfalls and process. IEEE Computer35(8):90—912002.
[24]莫斯里(Mosley,D.J.)等著. 邓波,黄丽娟,曹青春等译.软件测试自动化. 北京:机械工业出版社.2003.
[25]P.Stocks and D.Carrington. AFramework for Specication-based Testing.IEEETransactionsonSoftwareEngineering,1996,22(11):777~793
[26]Carl Nagle. Test Automation Frameworks Data Driven Test Automation:ForRationalRobotV2000.2000.
[27]Davis, Harold. Visual C#. NET Programming. Alameda, Calif :Sybex,2002.
[28]Price, Jason, MikeGunderloy. Mastering Visual C# .NET. Alameda,Calif:Sybex,2002.
[29]Troelson, Andrew. C# and the .NET Platform. Berkeley, Calif :Apress,2001.
[30]Pohl, Ira. C# by Dissection: The Essentials of C# Programming. Boston,Mass:Addison-Wesley,2002.
[31]Simon Robinson ,Christian Nagel等著.李敏波译.C#高级编程.北京:清华大学出版社,2005.
[32]Woei-Kae Chen, Tung-Hung Tsai, and Hung-Hsing Chao. Integration ofSpecification-Based and CR-Based Approaches for GUI Testing.Proceedings of the 19th International Conference on AdvancedInformationNetworkingandApplications(AINA'05)2005.
[2] Ron Patton.周予滨,姚静译.软件测试.北京.机械工业出版社.2002.23-25.
[3] B. A. Myers. User interface software tools. ACM Transactions onComputer-HumanInteraction2(1):64-1031995.
[4] Glenford J. Myers. The Art of Software Testing Second Edition. JohnWileyandSons.2004.
[5] LauraRose.GettingtheMostfromYourAutomatedTesting2001.
[6] AtifM.Memon.RegressionTestingofGUIs.ESEC/FSE'032003.
[7] Kanglin Li,Mengqi Wu. Effective GUI Test Automation Developing anAutomatedGUITestingTool.Sybex.2004.
[8] Steve McConnell. 10 Keys to Successful Software Projects AnExebuttonliveGuide.Tech.Presentation.2001.
[9] Boris Beizer. Software Testing Techniques. Van Nostrand reinholdPublishing.1983.
[10]古乐, 史林九著. 软件测试技术概论.北京:清华大学出版社.2004.
[11]惠特克J A著. 马良荔,俞立军译. 实用软件测试指南. 北京:电子工业出版社.2003.
[12]Atif M.Memon. Hierarchical GUI Test Case Generation UsingAutomationPlanning. IEEETransactions onSoftwareEngineering. 2001.27(2).
[13]Atif Memon, Ishan Banerjee,Adithya Nagarajan.GUI Ripping. ReverseEngineeringofGraphicalUserInterfacesforTesting.WCRE’03.2003.
[14]Juichi Takahashi, Yoshiaki Kakuda.Effective Automated Testing. ASolutionofGraphicalObjectVerification.ATS’02.2002.
[15]Atif M. Memon. A comprehensive framework for testing graphical userinterfacesUMIMicroform.2001.
[16]S. Rosaria, H. Robinson. Applying models in your testing process.InformationandSoftwareTechnology42(2000)815-824.
[17]MichaelKelly.Choosingatestautomationframework.2003.
[18]Mark Fewster, Dorothy Graham 著.舒智勇等译.软件测试自动化技术与实例详解.北京:电子工业出版社.2000.
[19]Tassey, Gregory. The Economic Impacts of Inadequate Infrastructure forSoftwareTesting.Collingdale,PA:DIANEPublishingCo,2003.
[20]Dustin, Elfriede. Effective Software Testing: 50 Specific Ways toImproveYourTesting.Boston,Mass:Addison-Wesley,2002.
[21](美)Charles F.Goldfarb, Paul Prescod著.XML实用技术.清华大学出版社,1999.9.
[22]Kanglin Li, Mengqi Wu. Effective Software Test Automation.Sybex.2004.
[23]Atif M. Memon. GUI testing: Pitfalls and process. IEEE Computer35(8):90—912002.
[24]莫斯里(Mosley,D.J.)等著. 邓波,黄丽娟,曹青春等译.软件测试自动化. 北京:机械工业出版社.2003.
[25]P.Stocks and D.Carrington. AFramework for Specication-based Testing.IEEETransactionsonSoftwareEngineering,1996,22(11):777~793
[26]Carl Nagle. Test Automation Frameworks Data Driven Test Automation:ForRationalRobotV2000.2000.
[27]Davis, Harold. Visual C#. NET Programming. Alameda, Calif :Sybex,2002.
[28]Price, Jason, MikeGunderloy. Mastering Visual C# .NET. Alameda,Calif:Sybex,2002.
[29]Troelson, Andrew. C# and the .NET Platform. Berkeley, Calif :Apress,2001.
[30]Pohl, Ira. C# by Dissection: The Essentials of C# Programming. Boston,Mass:Addison-Wesley,2002.
[31]Simon Robinson ,Christian Nagel等著.李敏波译.C#高级编程.北京:清华大学出版社,2005.
[32]Woei-Kae Chen, Tung-Hung Tsai, and Hung-Hsing Chao. Integration ofSpecification-Based and CR-Based Approaches for GUI Testing.Proceedings of the 19th International Conference on AdvancedInformationNetworkingandApplications(AINA'05)2005.