软件故障注入关键技术研究
|
摘要
可靠性是计算机系统的重要指标之一,在航天、航海、金融、交通、核工业控制等很多任务或时间关键性领域中,一旦在使用中出现故障后果不堪设想,可能造成巨大的经济损失和人员伤亡。因此对这些系统在投入使用之前一定要进行充分的测试,消除系统内可能存在的故障和漏洞,同时也要在使用之前模拟未来使用环境中可能出现的环境扰动,如常见的单粒子翻转故障,来对系统的可靠性等指标进行充分的评测,根据测试结果采用必要的设计和防护方法,保证其满足可靠性要求。
故障注入技术作为一种可靠性评测手段日益受到人们的重视。故障注入是通过人为地产生并引入故障到目标系统中加速系统产生错误和失效,然后通过分析故障引入后的系统行为反应,实现对目标系统可靠性进行评测的技术。与传统可靠性评测技术相比,它具有无需建立和求解复杂的系统模型,而且实验时间短、结果精度高等优点。故障注入技术有硬件实现的故障注入、模拟实现的故障注入和软件实现的故障注入三种。与前两种方法相比,基于软件实现的故障注入具有操作简便、开发成本低廉,不会对目标系统造成损伤等优点。本文主要研究了软件故障注入技术,将软件故障注入应用到软件测试的不同环节,分别研究了变异测试技术,基于中间层的故障注入技术与基于空间的故障注入技术,并设计了软件故障注入工具。
首先,根据各类故障模型利用突变理论对软件故障注入进行了建模及分析,并设计了故障注入策略算法。这种策略通过对故障模型的注入强度进行比较与分类,实现了根据注入强度进行故障注入实验的故障注入算法,为有效的故障注入实验实施提供依据。
研究讨论了面向程序变异测试技术的特点及发展规律,设计并提出了第一个关于Tcl的变异分析原型工具Tester,目前实现了3大类共13个运算符的变异算子,利用其进行了变异分析实验,实验结果标明Tester能有效的实施变异分析测试。研究了面向自动机的变异,面向自动机的变异主要目的是评测测试用例生成方法的有效性,这比面向程序的变异技术更进一步。我们设计了11种变异算子变异自动机,利用变异后的自动机评测测试用例生成方法,主要评测了TT方法与W方法。研究了面向CTL逻辑说明的变异,面向CTL逻辑说明的变异可以利用变异测试有效地生成测试用例。前面两者是利用变异测试评测测试用例的充分性及测试用例生成方法的有效性,此种变异则是利用变异测试直接生成测试用例,这里需要利用SMV模型检验器的反例生成能力。我们设计了10种变异算子变异CTL逻辑说明。这些我们都通过实验说明了提出的变异算子的有效性。
研究了基于中间层的软件故障注入。对于模块与环境之间的集成整合,Unix环境下研究了利用FIG实施的中间层故障注入技术。Windows环境下研究了利用detours库截获系统调用,对截获的调用注入故障,模拟各种环境扰动故障,通过这种方式评测模块面对可能环境故障的可恢复性与可靠性。对于模块之间的整合,研究利用软件故障注入实现的模块级和信号级的可靠性评测。目前,对于模块级的可靠性评测可以利用的理论有单调关联系统理论,本文对此理论作了介绍,指出了将其应用于利用软件故障注入评测系统可靠性方面的不足,对于此种情况提出了一种基于软件故障注入技术的可靠性评测方法。针对星载软件系统因宇宙射线和环境扰动而产生的软件错误及错误传播问题,研究了一种错误传播分析方法,利用此方法可从信号和模块两个层次评测软件的可靠性,并根据结果对系统信号或模块的脆弱性进行分析,找出系统较为脆弱的信号与模块及最可能传播错误的信号传播路径;定义了信号与模块的错误传播率、暴露率等参数,并设计了参数的计算方法;提出了错误传播图的构建算法。
研究了基于存储空间注入技术的软件故障注入实验与分析中存在的问题。提出了并设计了两种基于空间注入技术的注入方式,并利用它们分别进行了故障注入实验,通过实验着重分析了注入地址不同的空间分布对实验产生的影响,设计了实验记录格式并规范化了实验数据的分析和计算。完善了软件故障注入理论,提出了注入强度λ的概念,证明了所提出的两种注入方式的故障计数过程本质是泊松过程。详细讨论并分析了基于不同空间地址概率分布的故障注入实验问题,得出结论,对于空间注入技术实施的故障注入实验总存在一种相对较优的注入方案。这里研究内容可对利用软件故障注入对软件系统进行可靠性分析的工作提供帮助。
介绍了我们设计的单计算机软件故障注入工具SOFI,本章详细讨论了研究开发的基于软件故障注入技术的单计算机系统可靠性评测系统,包括该系统的主要功能模块及设计中值得注意的问题等,此外,还给出了应用该评测系统所进行的故障注入实验及可靠性评测实验,包括基于空间的注入实验和基于中间层的注入实验。实验表明了SOFI工具的灵活性与有效性。
Reliability is an important index of computer systems. In many task- or time- critical fields, such as aeronautics, navigation, finance, traffic, nuclear reactor monitoring etc., once there were some faults it may cause huge economic lose, or even personnel lose. So it is necessary to test these systems fully before they run in real environments. And it also needs to simulate environment disturbance such as single event upsets to evaluate reliability indices of computer system. Based on the indices and test results guarantee mechanism can be deployed effectively.
Fault injection as a method of evaluating reliability is becoming the research focus of software test. Traditional dependability evaluation methods are combinatorial model techniques and test techniques, which need to build and solve complex dependability model of the systems under test or spend much time for experiments. Different from these methods, the main advantage of fault injection is that there is no need to build any complex system model, no need to solve complex state equations and much less time consuming. Fault injection technique includes three kinds, which are hardware implemented fault injection, software implemented fault injection and simulation implemented fault injection. Compared with the other two injection techniques, software implemented fault injection has many advantages such as low development cost, good flexibility and no damage to target system and so on. In this paper the research focus is software implemented fault injection and it applies fault injection into different stages of software testing.
The model based on catastrophe theory is design to illustrate the behavior of fault injection and its influence on system .Through this model, many complicate relations are found out. By this model an injecting algorithm based on fault strength is designed.
Study the characteristic of mutation test and its developing trend. For program-based mutation a tool called Tester is designed to mutate Tcl language. Now Tester implemented 13 mutation operators and is used to do mutation experiments. The results of experiment prove the validity of Tester. For finite state machine mutation 11 mutation operators are designed to mutate finite state machine.
The goal of mutation experiments was evaluate effectiveness of methods that generate test cases. TT and W methods are evaluated. For CTL specification mutation 10 mutation operators are designed to mutate specifications. The goal of mutation experiments is to generate test cases under the help of SMV. SMV will produce counter-examples when CTL specification is false and these counter-examples can be very used as effective test cases. Study the software implemented fault injection based on middle layer injection.
Considering modules interact with system fault injection in glibc of Unix environment is studied. For Windows system fault injection experiments of Microsoft office products are implemented and results of experiments are analyzed. Considering modules interact with modules a method is designed to analyze error propagation rules and an implement algorithm is put forward in order to solve the problem, i.e. finding out the signals and modules that are more vulnerable to environmental disturbance and should be assigned errors detecting and correcting mechanisms or fault tolerance techniques firstly.
Based on memory space injecting technique two injecting methods were designed. Two kinds of injecting experiments were implemented according to these methods. By designing proper format of experiments records the reliability parameters were computed effectively. Define injecting strength and prove that the injecting count process is naturally a Poisson process. This proof is important because it means that random process theory can be introduced to study software-implemented fault injection. Usually fault injecting experiment is time-consuming and ineffective so it is important to choose an injecting plan that can optimize tests. The different distributions of location random variable X were used to implement experiments.
In the last of the dissertation, a detailed description on the dependability evaluation tool is given. It includes the general structure of the tool, the detailed design and implementation of the main modules and so on. The reliability experiments are implemented based on middle layer and space injection technique and reliability indices are computed.
故障注入技术作为一种可靠性评测手段日益受到人们的重视。故障注入是通过人为地产生并引入故障到目标系统中加速系统产生错误和失效,然后通过分析故障引入后的系统行为反应,实现对目标系统可靠性进行评测的技术。与传统可靠性评测技术相比,它具有无需建立和求解复杂的系统模型,而且实验时间短、结果精度高等优点。故障注入技术有硬件实现的故障注入、模拟实现的故障注入和软件实现的故障注入三种。与前两种方法相比,基于软件实现的故障注入具有操作简便、开发成本低廉,不会对目标系统造成损伤等优点。本文主要研究了软件故障注入技术,将软件故障注入应用到软件测试的不同环节,分别研究了变异测试技术,基于中间层的故障注入技术与基于空间的故障注入技术,并设计了软件故障注入工具。
首先,根据各类故障模型利用突变理论对软件故障注入进行了建模及分析,并设计了故障注入策略算法。这种策略通过对故障模型的注入强度进行比较与分类,实现了根据注入强度进行故障注入实验的故障注入算法,为有效的故障注入实验实施提供依据。
研究讨论了面向程序变异测试技术的特点及发展规律,设计并提出了第一个关于Tcl的变异分析原型工具Tester,目前实现了3大类共13个运算符的变异算子,利用其进行了变异分析实验,实验结果标明Tester能有效的实施变异分析测试。研究了面向自动机的变异,面向自动机的变异主要目的是评测测试用例生成方法的有效性,这比面向程序的变异技术更进一步。我们设计了11种变异算子变异自动机,利用变异后的自动机评测测试用例生成方法,主要评测了TT方法与W方法。研究了面向CTL逻辑说明的变异,面向CTL逻辑说明的变异可以利用变异测试有效地生成测试用例。前面两者是利用变异测试评测测试用例的充分性及测试用例生成方法的有效性,此种变异则是利用变异测试直接生成测试用例,这里需要利用SMV模型检验器的反例生成能力。我们设计了10种变异算子变异CTL逻辑说明。这些我们都通过实验说明了提出的变异算子的有效性。
研究了基于中间层的软件故障注入。对于模块与环境之间的集成整合,Unix环境下研究了利用FIG实施的中间层故障注入技术。Windows环境下研究了利用detours库截获系统调用,对截获的调用注入故障,模拟各种环境扰动故障,通过这种方式评测模块面对可能环境故障的可恢复性与可靠性。对于模块之间的整合,研究利用软件故障注入实现的模块级和信号级的可靠性评测。目前,对于模块级的可靠性评测可以利用的理论有单调关联系统理论,本文对此理论作了介绍,指出了将其应用于利用软件故障注入评测系统可靠性方面的不足,对于此种情况提出了一种基于软件故障注入技术的可靠性评测方法。针对星载软件系统因宇宙射线和环境扰动而产生的软件错误及错误传播问题,研究了一种错误传播分析方法,利用此方法可从信号和模块两个层次评测软件的可靠性,并根据结果对系统信号或模块的脆弱性进行分析,找出系统较为脆弱的信号与模块及最可能传播错误的信号传播路径;定义了信号与模块的错误传播率、暴露率等参数,并设计了参数的计算方法;提出了错误传播图的构建算法。
研究了基于存储空间注入技术的软件故障注入实验与分析中存在的问题。提出了并设计了两种基于空间注入技术的注入方式,并利用它们分别进行了故障注入实验,通过实验着重分析了注入地址不同的空间分布对实验产生的影响,设计了实验记录格式并规范化了实验数据的分析和计算。完善了软件故障注入理论,提出了注入强度λ的概念,证明了所提出的两种注入方式的故障计数过程本质是泊松过程。详细讨论并分析了基于不同空间地址概率分布的故障注入实验问题,得出结论,对于空间注入技术实施的故障注入实验总存在一种相对较优的注入方案。这里研究内容可对利用软件故障注入对软件系统进行可靠性分析的工作提供帮助。
介绍了我们设计的单计算机软件故障注入工具SOFI,本章详细讨论了研究开发的基于软件故障注入技术的单计算机系统可靠性评测系统,包括该系统的主要功能模块及设计中值得注意的问题等,此外,还给出了应用该评测系统所进行的故障注入实验及可靠性评测实验,包括基于空间的注入实验和基于中间层的注入实验。实验表明了SOFI工具的灵活性与有效性。
Reliability is an important index of computer systems. In many task- or time- critical fields, such as aeronautics, navigation, finance, traffic, nuclear reactor monitoring etc., once there were some faults it may cause huge economic lose, or even personnel lose. So it is necessary to test these systems fully before they run in real environments. And it also needs to simulate environment disturbance such as single event upsets to evaluate reliability indices of computer system. Based on the indices and test results guarantee mechanism can be deployed effectively.
Fault injection as a method of evaluating reliability is becoming the research focus of software test. Traditional dependability evaluation methods are combinatorial model techniques and test techniques, which need to build and solve complex dependability model of the systems under test or spend much time for experiments. Different from these methods, the main advantage of fault injection is that there is no need to build any complex system model, no need to solve complex state equations and much less time consuming. Fault injection technique includes three kinds, which are hardware implemented fault injection, software implemented fault injection and simulation implemented fault injection. Compared with the other two injection techniques, software implemented fault injection has many advantages such as low development cost, good flexibility and no damage to target system and so on. In this paper the research focus is software implemented fault injection and it applies fault injection into different stages of software testing.
The model based on catastrophe theory is design to illustrate the behavior of fault injection and its influence on system .Through this model, many complicate relations are found out. By this model an injecting algorithm based on fault strength is designed.
Study the characteristic of mutation test and its developing trend. For program-based mutation a tool called Tester is designed to mutate Tcl language. Now Tester implemented 13 mutation operators and is used to do mutation experiments. The results of experiment prove the validity of Tester. For finite state machine mutation 11 mutation operators are designed to mutate finite state machine.
The goal of mutation experiments was evaluate effectiveness of methods that generate test cases. TT and W methods are evaluated. For CTL specification mutation 10 mutation operators are designed to mutate specifications. The goal of mutation experiments is to generate test cases under the help of SMV. SMV will produce counter-examples when CTL specification is false and these counter-examples can be very used as effective test cases. Study the software implemented fault injection based on middle layer injection.
Considering modules interact with system fault injection in glibc of Unix environment is studied. For Windows system fault injection experiments of Microsoft office products are implemented and results of experiments are analyzed. Considering modules interact with modules a method is designed to analyze error propagation rules and an implement algorithm is put forward in order to solve the problem, i.e. finding out the signals and modules that are more vulnerable to environmental disturbance and should be assigned errors detecting and correcting mechanisms or fault tolerance techniques firstly.
Based on memory space injecting technique two injecting methods were designed. Two kinds of injecting experiments were implemented according to these methods. By designing proper format of experiments records the reliability parameters were computed effectively. Define injecting strength and prove that the injecting count process is naturally a Poisson process. This proof is important because it means that random process theory can be introduced to study software-implemented fault injection. Usually fault injecting experiment is time-consuming and ineffective so it is important to choose an injecting plan that can optimize tests. The different distributions of location random variable X were used to implement experiments.
In the last of the dissertation, a detailed description on the dependability evaluation tool is given. It includes the general structure of the tool, the detailed design and implementation of the main modules and so on. The reliability experiments are implemented based on middle layer and space injection technique and reliability indices are computed.
引文
[1] Shirvani Philip P. Fault-Tolerant Computing for Radiation Environments[R]. Stanford: Stanford University, 2001: 1–24.
[2] Dodd P E, Massengill L W. Basic Mechanisms and Modeling of Single-Event Upset in Digital Microelectronics. IEEE TRANSACTION ON NUCLEAR SCIENCE, 2003, 50(3):583-602.
[3] Monson J S, Wirthlin M, Hutchings B. Fault Injection Results of Linux Operating on an FPGA Embedded Platform[C]//Proc. of 2010 International Conference on Reconfigurable Computing and FPGAs. Mexico: IEEE CS, 2010: 37-42.
[4] Timothy C M, Murray H W. A New Physical Mechanism for Soft Errors in Dynamic Memories[C]//Proc. of the 16th Reliability Physics Symposium. San Diego: IEEE CS, 1978: 33-40.
[5] Baumann R C. Radiation-induced Soft Errors in Advanced Semiconductor Technologies[J]. IEEE Transactions on Device and Materials Reliability, 2005, 5(3):305–316.
[6] Barbosa R, Silva N, Duraes J, Madeira H. Verification and Validation of (Real Time) COTS Products Using Fault Injection Techniques[C]//Proc. of the 6th Int’l IEEE Conf. on Commercial-off-the-Shelf (COTS)-Based Software Systems (ICCBSS 2007). Los Alamitos: IEEE CS, 2007: 233?242.
[7] Lopez-Ongil C, Entrena L, Garcia-Valderas, et al. A Unified Environment for Fault Injection at Any Design Level Based on Emulation[J]. IEEE Trans. on Nuclear Science, 2007, 54(4): 946-950.
[8] Blanc S, Gracia J, Gil P J. A Fault Hypothesis Study on the TTP/C Using VHDL-Based and Pin-Level Fault Injection Techniques[C]//Proc. of the 17th IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems. Los Alamitos: IEEE CS, 2002: 254-262.
[9] Case G R. Analysis of Actual Fault Mechanism in CMOS Logic Gates[C]//Proceedings of the 13th Design Automation Conference. 1976: 265-270
[10] Van de Goor A J, Verruijt C A. An Overview of Deterministic FunctionalRAMChip Testing[J]. ACM Compute Surv, 1990, 22(1): 5-33.
[11]孙峻朝,王建莹,杨孝宗.管脚级故障模型的分析与生成技术的研究[J].计算机学报,1999,22(8):845-851.
[12]李华旺,刘海涛.航天单粒子事件故障注入系统研究[J].量子电子学报,2002, 19(1):57-60.
[13] Ghosh S, Kelly J L. Bytecode Fault Injection for Java Software[J]. The Journal of Systems and Software, 2008, 47(2):1-10.
[14] Gaisler J. Evaluation of a 32-bit Microprocessor with Built-in Concurrent. Error-Detection[C]//Proc. of 27th IEEE Int. Symp. on Fault Tolerant Computing. WA:Seattle, 1997: 42-46.
[15] Jeitler M, Delvai M, Reichor S, et al. FuSE - a Hardware Accelerated HDL Fault Injection Tool[C]//Proc. of the 5th Southern Conference on Programmable Logic. Sao Carlos: IEEE CS, 2009:89-94.
[16] Sharma A, Singh B. Simulation of Fault Injection of Microprocessor System Using VLSI Architecture System[C]//Proc. of TENCON 2009 - 2009 IEEE Region 10 Conference. Singapore: IEEE CS, 2009:1-5.
[17] Arlat J, Aguera M, Amat L, et al. Fault Injection for DependabilityValidation: a Methodology and Some Applications[J]. IEEE Transactions on Software Engineering, 1990, 16(2):166–182.
[18] Gunneflo U, Karlsson J, Torin J.FIST-A Fault Injection System for the Study of Fault Effects on Computers[R]. Sweden: Chalmers University of Technology, 1990: Tech. Report No. 91.
[19] Madeira H, Rela M, Moreira F, et al. Rifle: a General Purpose Pin-level Fault Injector[C]//Proc. EDCC-1. Berlin, 1994: 199–216.
[20] Karlsson J, Folkesson P, Arlat J, et al. Evaluation of the MARS Fault Tolerance Mechanisms Using Three Physical Fault Injection Techniques[C]// Third Int'l Workshop on Integrating Error Models with Fault Injection.1994: 21-22.
[21] Sass Ron, Sharma R R, DeBardeleben N. Towards a hardware fault-injection testbed to support reproducible resiliency experiments[C]//Proc. of the 2009 workshop on Resiliency in high performance. New York: ACM, 2009: 15-22.
[22] Mahmood A, McClushey E J. Concurrent Error Detection Using WatchdogProcessors- a Survey[J]. IEEE Transaction on Computers, 1988, 37(2): 160-174.
[23] Abadi Martin, Budiu Mihai, Erlingsson Ulfar, et al. Control-flow Integrity Principles, Implementations, and Applications[J]. ACM Transactions on Information and System Security, 2009,13(1): 1-41.
[24] di Sanzo P, Ciciani B, Quaglia F, et al. A Performance Model of Multi-Version Concurrency Control[C]//Proc. of IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems. Baltimore: IEEE CS, 2008:1-10.
[25] Junsung K, Lakshmanan K, Rajkumar R. R-BATCH: Task Partitioning for Fault-tolerant Multiprocessor Real-Time Systems[C]//Proc. of 2010 IEEE 10th International Conference on Computer and Information Technology. Bradford: IEEE CS, 2010: 1872-1879.
[26] Mavis D G, Eaton P H, Sibley M D. Multiple Bit Upsets and Error Mitigation in Ultra-Deep Submicron SRAMS[J]. IEEE Transactions on Nuclear Science, 2008, 55(6):3288-3294.
[27] Espinosa-Duran J M, Trujillo-Olaya V, Velasco-Medina J. Bit-flip injection strategies for FSMs modeled in VHDL behavioral level[C]//Proc. of the 11th Latin American Test Workshop. Pule del Este: IEEE CS, 2010: 1-5.
[28] Gracia-Moran J, Gil-Tomas D, Saiz-Adalid L J. Experimental Validation of a Fault Tolerant Microcomputer System against Intermittent Faults[C]//Proc. of 2010 IEEE/IFIP International Conference on Dependable Systems and Networks. Chicago: IEEE CS, 2010: 413-418.
[29] Carreira J, Madeira H, Silva J G. Xception: Software Fault Injection and Monitoring in Processor Functional Units[J]. IEEE Trans on Software Engineering, 1998, 24(2):1–25.
[30] Young L T, Iyer R K, Goswami K K, et al. A Hybrid Monitor Assisted Fault Injection Environment[C]//The Third IFIP Working Confernce on Dependable Computing for Critical Application. 1992.
[31] Kanawati G A, Kanawati N A, Abraham J A. FERRARI: A Flexible Software-Based Fault and Error Injection System[J]. IEEE Transactions on Computer, 1995, 44(2): 248-260.
[32] Goldberg I, Wagner D, Thomas R, et al. A Secure Environment for UntrustedHelper Applications: Confining the Wily Hacker[C]//Proceedings of the 6th Usenix Security Symposium. San Jose: USENIX, 1996: 1-13.
[33] Dawson S, Jahanian F, Mitton T. ORCHESTRA: A Probing and Fault Injection Environment for Testing Protocol Implementions[C]//Proceedings of IEEE international Computer Performance and Dependability Symposium. 1996: 56.
[34] Tsai T K, Iyer R K. Measuring Fault Tolerance with the FTAPE Fault Injection Tool[C]//Proceedings of the Eighth International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1995: 26-40.
[35] Kropp N P, Koopman P J, Siewiorek D P. Automated Robustness Testing of Off-the Shelf Software Components[C]//Proceedings of the 28th International Symposium on Fault-Tolerant Computing. 1998:464-468.
[36] Miller B P, Fredriksen L, So B. An Empirical Study of the Reliability of UNIX Utilities[J]. Communications of the ACM, 1990, 33(12):32-44.
[37] Center for Software Engineering Research[OL]. Holodeck. http://se.fit.edu/ holodeck/, 2002.
[38] Broadwell P, Sastry N, Traupman J. FIG: Fault Injection in glibc[C]//In Workshop on Self-Healing, Adaptive, and Self-Managed Systems (SHAMAN), New York, 2002.
[39]王胜文.基于软件的故障注入方法研究[D].哈尔滨:哈尔滨工业大学学位论文,2005:28-29.
[40]荣盘祥.复杂系统脆性理论及其理论框架的研究[D].哈尔滨:哈尔滨工程大学学位论文,2006:1-50.
[41]凌复华.突变理论及其应用[M].上海:上海交通大学出版社,1987:1-100.
[42] Zhao L, Tang M K. Dynamic Risk Evaluation of Geological Catastrophe of Tailing Reservoir Based on Catastrophe Theory[C]//Proc. of 4th International Conference on Bioinformatics and Biomedical Engineering. Chengdu: IEEE CS, 2010:1-4.
[43] Oliveira G M, Cechin S L, Weber T S. Dependability Evaluation of Distributed Systems through Partitioning Fault Injection[C]//Proc. of 11th Latin American Test Workshop. Pule del Este: IEEE CS, 2010:1-6.
[44] Schmid M, Ghosh A, Hill F. Techniques for Evaluating the Robustness of Windows NT software[C]//Proc. of the 2000 DARPA InformationSurvivability Conf. and Exposition. Los Alamitos: IEEE Computer Society, 2000:347?360.
[45] Aidemark J, Vinter J , Folkesson P, et al. GOOFI: Generic Object-oriented Fault Injection Tool[C]//Proc. of the Int’l Conf. on Dependable Systems and Networks. Los Alamitos: IEEE Computer Society, 2001: 83?88.
[46] Kao W I, Iyer R K, Tang D. FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX system Behavior under Faults. IEEE Trans. on Software Engineering, 1993, 19(11):1105?1118.
[47] Some R R, Kim W S, Khanoyan G, et al. A Software-implemented Fault Injection Methodology for Design and Validation of System Fault Tolerance[C]//Proc. of the 2001 International Conference on Dependable Systems and Networks. Los Alamitos: IEEE Computer Society, 2001: 501-506.
[48] Some R R, Kim W S, Khanoyan G, et al. Fault Injection Experiment Results in Space Borne Parallel Application Programs[C]//Proc. of IEEE Aerospace Conference. Los Alamitos: IEEE Computer Society, 2002: 2133-2147.
[49]陈锦富,卢炎生,谢晓东.软件错误注入测试技术研究[J].软件学报,2009, 20(6):1425-1433.
[50]彭俊杰.基于软件故障注入的星载系统可靠性评测[D].哈尔滨:哈尔滨工业大学学位论文,2005:20-21.
[51] Li J H, Dai G X, Li H H. Mutation Analysis for Testing Finite State Machines[C]//Proc. of ISECS '09. Second International Symposium on Electronic Commerce and Security. Nanchang: IEEE CS, 2009: 620-624.
[52] Chow T S. Testing Software Design Modelled by Finite State Machines[J]. IEEE Transactions on Software Engineering, 1978, 4(3):178-187.
[53] Fabbri S C P F, Delamaro M E, Maldonado J C, et al. Mutation Analysis Testing for Finite State Machines[C]//Proceedings of the fifth International Symposium on Software Reliability Engineering. Los Alamitos,CA: IEEE Press, 1994: 220-229.
[54] Black P E, Okun V, Yesha Y. Mutation of Model Checker Specification for Test Generation and Evaluation[C]//Proceedings of Mutation 2000. San Jose: California, 2000:14-20.
[55] Schuler D, Dallmeier V, Zeller A. Efficient Mutation Testing by Checking Invariant Violations[C]//Proc. of the eighteenth international symposium onSoftware testing and analysis. New York: ACM, 2009: 69-80.
[56] Delamaro M E, Maldonado J C. Interface Mutation: Assessing Testing Quality at Interprocedural Level[C]//Proceedings of the 19th International Conference of the Chilean Computer Science Society (SCCC’99).Talca, Chile,1999 : 78–86.
[57] Batth S S, Vieira E R, Cavalli A R, et al. Specification of Timed EFSM Fault Models in SDL[C]//Proceedings of the 27th IFIP WG 6.1 International Conference on Formal Techniques for Networked and Distributed Systems (FORTE’07). Tallinn, Estonia: Springer, 2007: 50–65.
[58] Fraser G, Wotawa F. Mutant Minimization for Model-Checker Based Test-Case Generation[C]//Proceedings of the 3rd Workshop on Mutation Analysis (MUTATION’07). Windsor, UK: IEEE Computer Society, 2007: 161–168.
[59] Souza S D R S D, Maldonado J C, Fabbri S C P F, et al. Mutation Testing Applied to Estelle Specifications[J]. Software Quality Control, 1999, 8(4):285-301.
[60] Fabbri S C P F, Maldonado J C, Masiero P C, et al. Mutation Testing Applied to Validate Specifications Based on Petri Nets[C]//Proceedings of the IFIP TC6 8th International Conference on Formal Description Techniques. 1995:329–337.
[61] Jing C, Wang Z, Shi X, et al. Mutation Testing of Protocol Messages Based on Extended TTCN-3[C]//Proceedings of the 22nd International Conference on Advanced Information Networking and Applications (AINA’08). Okinawa, Japan, 2008 :667–674.
[62] Le Traon Y, Mouelhi T, Baudry B. Testing Security Policies: Going Beyond Functional Testing[C]//The 18th IEEE International Symposium on Software Reliability. Sweden: IEEE Computer Society, 2007: 93–102.
[63] Lee S, Bai X, Chen Y. Automatic Mutation Testing and Simulation on OWL-S Specified Web Services[C]//Proceedings of the 41st Annual Simulation Symposium (ANSS’08).Ottawa, Canada, 2008:149–156.
[64] Jia Y, Harman M. An Analysis and Survey of the Development of Mutation Testing[J]. IEEE Transaction on Software Engineering, 2010.
[65] Acree A T, Budd T A, DeMillo R A, et al. Mutation Analysis[R].Georgia :Georgia Institute of Technology, 1979: GIT-ICS-79/08.
[66] DeMillo R A, Lipton R J, Sayward F G. Hints on Test Data Selection: Help for the Practicing Programmer[J]. Computer, 1978, 11(4): 34–41.
[67] Offutt A. The Coupling Effect: Fact or Fiction[C]//Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification. Key west, Florida, United States, 1989:131-140.
[68] Lipton R J, Sayward F G. The Status of Research on Program Mutation[C]//Proceedings of the Workshop on Software Testing and Test Documentation. 1978: 355–373.
[69] Offutt A J. Investigations of the Software Testing Coupling Effect[J]. ACM Transactions on Software Engineering and Methodology, 1992, 1(1):5–20.
[70] Kapoor K. Formal Analysis of Coupling Hypothesis for Logical Faults[J]. Innovations in Systems and Software Engineering, 2006, 2(2):80–87.
[71] AMMANN P, Offutt J. Instruction to Software Testing. Cambridge University Press. 2008, 180-181
[72] King K N, Offutt A J. A Fortran Language System for Mutation-Based Software Testing[J]. Software:Practice and Experience, 1991, 21(7):685-718.
[73] Jia Y, Harman M. MILU: A Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language[C]//Proceedings of the 3rd Testing: Academic and Industrial Conference Practice and Research Techniques (TAIC PART’08). Windsor, UK: IEEE Computer Society, 2008:94–98.
[74] Simao A, Maldonado J C, da Silva Bigonha R. A Transformational Language for Mutant Description[J]. Computer Languages, Systems & Structures, 2009, 35(3): 322–339.
[75] Budd T A, Angluin D. Two Notions of Correctness and Their Relation to Testing[J]. Acta Informatica, 1982,18(1):31–45.
[76] Grun B J M, Schuler D, Zeller A. The Impact of Equivalent Mutants[C]// Proceedings of the 4th International Workshop on Mutation Analysis (MUTATION’09). Denver, Colorado: IEEE Computer Society, 2009:192–199.
[77] Baldwin D, Sayward F G. Heuristics for Determining Equivalence of Program Mutations[R]. Yale University, New Haven, Connecticut, Research Report 276, 1979.
[78] Offutt A J, Craft W M. Using Compiler Optimization Techniques to Detect Equivalent Mutants[J]. Software Testing, Verification and Reliability, 1994, 4(3):131–154.
[79] Offutt A J, Pan J. Detecting Equivalent Mutants and the Feasible Path Problem[C]//Proceedings of the 1996 Annual Conference on Computer Assurance. Gaithersburg, Maryland: IEEE Computer Society Press, 1996: 224–236.
[80] Voas J, McGraw G. Software Fault Injection: Inoculating Programs Against Errors[M]. USA: John Wiley & Sons, 1997: 1-100.
[81] Hierons R M, Harman M, Danicic S. Using Program Slicing to Assist in the Detection of Equivalent Mutants[J]. Software Testing, Verification and Reliability, 1999:9(4):233–262.
[82] Harman M, Hierons R, Danicic S. The Relationship between Program Dependence and Mutation Analysis[C]//Proceedings of the 1st Workshop on Mutation Analysis (MUTATION’00).San Jose: IEEE CS, 2001:5–13.
[83] Adamopoulos K, Harman M, Hierons R M. How to Overcome the Equivalent Mutant Problem and Achieve Tailored Selective Mutation Using Co-evolution[C]//Proceedings of the Genetic and Evolutionary Computation Conference. Seattle, Washington, USA: Springer, 2004: 1338–1349.
[84] Ellims M, Ince D C, Petre M. The Csaw C Mutation Tool: Initial Results[C]// Proceedings of the 3rd Workshop on Mutation Analysis (MUTATION’07).. Windsor, UK: IEEE Computer Society, 2007: 185–192.
[85] Welch B B, Jones K, Hobbs J. Practical Programming in Tcl and Tk[M]. Prentice Hall PTR, 2003:1-50.
[86] Mathur A P. Foundations of Software Testing[M]. Pearson Education, 2008:200-250.
[87] McMillan K L. Symbolic Model Checking[M]. Kluwer Academic Publishers, 1993:1-10.
[88] Clarke E M, Jr., Emerson E A , et al. Automatic Verification of Finite-state Concurrent Systems Using Temporal Logic Specifications[J]. ACM Transactions on Programming Languages and Systems, 1986, 8(2):244-263.
[89] Mealy, George H. A Method for Synthesizing Sequential Circuits[J]. Bell Systems Technical Journal, 1955:1045–1079.
[90] Bryant R E. Symbolic Boolean Manipulation with Ordered Binary-decision diagrams[J]. ACM Computing Surveys, 1992, 24(3): 293-318.
[91] Clarke, Grumberg , Peled. Model Checking[M]. The MIT Press, 1999: 14-14.
[92] Ammann P E, Black P E, Majurski W. Using Model Checking to Generate Tests from Specification[C]//Proceedings of Second International Conference on Formal Engineering Methods. 1998: 46-54.
[93] Hunt G, Brubacher D. Detours: binary interception of Win32 functions[C]//Proceedings of the 3rd conference on USENIX Windows NT Symposium. 1999:14-14.
[94] Martin Hiller. A Software Profiling Methodology for Design and Assessment of Dependable Software[D].Sweden: Chalmers University of Technology Ph.D. Thesis, 2002:50-70.
[95] Detours函数介绍[OL]. http://www.cnblogs.com/flying_bat/archive/ 2008/ 04/ 18/1159996.html.
[96] Barlow R E, Proschan F. Mathematical Theory of Reliability[M]. New York: Wiely, 1965.
[97] Barlow R E, Proschan F. Statistical Theory of Reliability and Life Testing[M]. New York: Holt Rinehart and Winston, 1975.
[98]曹晋华,程侃.可靠性数学引论[M].北京:高等教育出版社, 2006: 136-148.
[99] Han S, Shin K, Rosenberg H. DOCTOR: An Integrated Software Fault Injection Environment for Distributed Real-Time Systems[C]//In Int. Computer Performance and Dependability Symp. Erlangen, Germany, 1995: 204–213.
[100]Baldini A, Benso A, Chiusano S, et al.“BOND”: An interposition agents based fault injector for Windows NT[C]//Proc. of the 2000 IEEE Int’l Symp. on Defect and Fault Tolerance in VLSI Systems. Los Alamitos: IEEE CS, 2000: 387?395.
[101]Sanda P N, Kellington J W, Kudva P. Soft-error resilience of the IBM POWER6 processor[J].IBM Journal of Research and Development. 2008, 52(3):275-284.
[102]何书元.随机过程[M].北京:北京大学出版社,2008:25-31.
[103]Li P, Park H, Gao D, et al. Bridging the Gap between Data-Flow and Control-Flow Analysis for Anomaly Detection[C]//Proc. of 2008 AnnualComputer Security Applications Conference. IEEE CS, 2008:392-401.
[104]Dawson S, Jahanian F, Mitton T. A Software Fault Injection Tool on Real-time Mach[C]//Proceedings of the 16th IEEE Real-Time Systems Symposium. 1995: 130-140.
[105]Stott D T, Floering B, Burke D, et al. Nftape: a Framework for Assessing Dependability in Distributed Systems with Lightweight Fault Injectors[C]//Int. Computer Performance and Dependability Symp. Chicago IL, 2000: 91–100.
[106]Eric Wong W, Yu Qi, Lei Zhao, et al. Effective Fault Localization using Code Coverage[C]//Proc. of the 31st Annual International Computer Software and Applications Conference. Beijing: IEEE CS, 2007: 449-456.
[107]Torsten Lorentzen. Attitude Control and Determination System for DTUsat1[OL].2002.http://microsat.sm.bmstu.ru/e-library/Algorithms/CommonDesign/dtu_torsten.pdf
[108]Felix R. Hoots and Ronald L. Roehrich. Space track report no.3 [OL].1980.http://celestrack.com/NORAD/documentation/spacetrk.pdf.
[2] Dodd P E, Massengill L W. Basic Mechanisms and Modeling of Single-Event Upset in Digital Microelectronics. IEEE TRANSACTION ON NUCLEAR SCIENCE, 2003, 50(3):583-602.
[3] Monson J S, Wirthlin M, Hutchings B. Fault Injection Results of Linux Operating on an FPGA Embedded Platform[C]//Proc. of 2010 International Conference on Reconfigurable Computing and FPGAs. Mexico: IEEE CS, 2010: 37-42.
[4] Timothy C M, Murray H W. A New Physical Mechanism for Soft Errors in Dynamic Memories[C]//Proc. of the 16th Reliability Physics Symposium. San Diego: IEEE CS, 1978: 33-40.
[5] Baumann R C. Radiation-induced Soft Errors in Advanced Semiconductor Technologies[J]. IEEE Transactions on Device and Materials Reliability, 2005, 5(3):305–316.
[6] Barbosa R, Silva N, Duraes J, Madeira H. Verification and Validation of (Real Time) COTS Products Using Fault Injection Techniques[C]//Proc. of the 6th Int’l IEEE Conf. on Commercial-off-the-Shelf (COTS)-Based Software Systems (ICCBSS 2007). Los Alamitos: IEEE CS, 2007: 233?242.
[7] Lopez-Ongil C, Entrena L, Garcia-Valderas, et al. A Unified Environment for Fault Injection at Any Design Level Based on Emulation[J]. IEEE Trans. on Nuclear Science, 2007, 54(4): 946-950.
[8] Blanc S, Gracia J, Gil P J. A Fault Hypothesis Study on the TTP/C Using VHDL-Based and Pin-Level Fault Injection Techniques[C]//Proc. of the 17th IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems. Los Alamitos: IEEE CS, 2002: 254-262.
[9] Case G R. Analysis of Actual Fault Mechanism in CMOS Logic Gates[C]//Proceedings of the 13th Design Automation Conference. 1976: 265-270
[10] Van de Goor A J, Verruijt C A. An Overview of Deterministic FunctionalRAMChip Testing[J]. ACM Compute Surv, 1990, 22(1): 5-33.
[11]孙峻朝,王建莹,杨孝宗.管脚级故障模型的分析与生成技术的研究[J].计算机学报,1999,22(8):845-851.
[12]李华旺,刘海涛.航天单粒子事件故障注入系统研究[J].量子电子学报,2002, 19(1):57-60.
[13] Ghosh S, Kelly J L. Bytecode Fault Injection for Java Software[J]. The Journal of Systems and Software, 2008, 47(2):1-10.
[14] Gaisler J. Evaluation of a 32-bit Microprocessor with Built-in Concurrent. Error-Detection[C]//Proc. of 27th IEEE Int. Symp. on Fault Tolerant Computing. WA:Seattle, 1997: 42-46.
[15] Jeitler M, Delvai M, Reichor S, et al. FuSE - a Hardware Accelerated HDL Fault Injection Tool[C]//Proc. of the 5th Southern Conference on Programmable Logic. Sao Carlos: IEEE CS, 2009:89-94.
[16] Sharma A, Singh B. Simulation of Fault Injection of Microprocessor System Using VLSI Architecture System[C]//Proc. of TENCON 2009 - 2009 IEEE Region 10 Conference. Singapore: IEEE CS, 2009:1-5.
[17] Arlat J, Aguera M, Amat L, et al. Fault Injection for DependabilityValidation: a Methodology and Some Applications[J]. IEEE Transactions on Software Engineering, 1990, 16(2):166–182.
[18] Gunneflo U, Karlsson J, Torin J.FIST-A Fault Injection System for the Study of Fault Effects on Computers[R]. Sweden: Chalmers University of Technology, 1990: Tech. Report No. 91.
[19] Madeira H, Rela M, Moreira F, et al. Rifle: a General Purpose Pin-level Fault Injector[C]//Proc. EDCC-1. Berlin, 1994: 199–216.
[20] Karlsson J, Folkesson P, Arlat J, et al. Evaluation of the MARS Fault Tolerance Mechanisms Using Three Physical Fault Injection Techniques[C]// Third Int'l Workshop on Integrating Error Models with Fault Injection.1994: 21-22.
[21] Sass Ron, Sharma R R, DeBardeleben N. Towards a hardware fault-injection testbed to support reproducible resiliency experiments[C]//Proc. of the 2009 workshop on Resiliency in high performance. New York: ACM, 2009: 15-22.
[22] Mahmood A, McClushey E J. Concurrent Error Detection Using WatchdogProcessors- a Survey[J]. IEEE Transaction on Computers, 1988, 37(2): 160-174.
[23] Abadi Martin, Budiu Mihai, Erlingsson Ulfar, et al. Control-flow Integrity Principles, Implementations, and Applications[J]. ACM Transactions on Information and System Security, 2009,13(1): 1-41.
[24] di Sanzo P, Ciciani B, Quaglia F, et al. A Performance Model of Multi-Version Concurrency Control[C]//Proc. of IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems. Baltimore: IEEE CS, 2008:1-10.
[25] Junsung K, Lakshmanan K, Rajkumar R. R-BATCH: Task Partitioning for Fault-tolerant Multiprocessor Real-Time Systems[C]//Proc. of 2010 IEEE 10th International Conference on Computer and Information Technology. Bradford: IEEE CS, 2010: 1872-1879.
[26] Mavis D G, Eaton P H, Sibley M D. Multiple Bit Upsets and Error Mitigation in Ultra-Deep Submicron SRAMS[J]. IEEE Transactions on Nuclear Science, 2008, 55(6):3288-3294.
[27] Espinosa-Duran J M, Trujillo-Olaya V, Velasco-Medina J. Bit-flip injection strategies for FSMs modeled in VHDL behavioral level[C]//Proc. of the 11th Latin American Test Workshop. Pule del Este: IEEE CS, 2010: 1-5.
[28] Gracia-Moran J, Gil-Tomas D, Saiz-Adalid L J. Experimental Validation of a Fault Tolerant Microcomputer System against Intermittent Faults[C]//Proc. of 2010 IEEE/IFIP International Conference on Dependable Systems and Networks. Chicago: IEEE CS, 2010: 413-418.
[29] Carreira J, Madeira H, Silva J G. Xception: Software Fault Injection and Monitoring in Processor Functional Units[J]. IEEE Trans on Software Engineering, 1998, 24(2):1–25.
[30] Young L T, Iyer R K, Goswami K K, et al. A Hybrid Monitor Assisted Fault Injection Environment[C]//The Third IFIP Working Confernce on Dependable Computing for Critical Application. 1992.
[31] Kanawati G A, Kanawati N A, Abraham J A. FERRARI: A Flexible Software-Based Fault and Error Injection System[J]. IEEE Transactions on Computer, 1995, 44(2): 248-260.
[32] Goldberg I, Wagner D, Thomas R, et al. A Secure Environment for UntrustedHelper Applications: Confining the Wily Hacker[C]//Proceedings of the 6th Usenix Security Symposium. San Jose: USENIX, 1996: 1-13.
[33] Dawson S, Jahanian F, Mitton T. ORCHESTRA: A Probing and Fault Injection Environment for Testing Protocol Implementions[C]//Proceedings of IEEE international Computer Performance and Dependability Symposium. 1996: 56.
[34] Tsai T K, Iyer R K. Measuring Fault Tolerance with the FTAPE Fault Injection Tool[C]//Proceedings of the Eighth International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1995: 26-40.
[35] Kropp N P, Koopman P J, Siewiorek D P. Automated Robustness Testing of Off-the Shelf Software Components[C]//Proceedings of the 28th International Symposium on Fault-Tolerant Computing. 1998:464-468.
[36] Miller B P, Fredriksen L, So B. An Empirical Study of the Reliability of UNIX Utilities[J]. Communications of the ACM, 1990, 33(12):32-44.
[37] Center for Software Engineering Research[OL]. Holodeck. http://se.fit.edu/ holodeck/, 2002.
[38] Broadwell P, Sastry N, Traupman J. FIG: Fault Injection in glibc[C]//In Workshop on Self-Healing, Adaptive, and Self-Managed Systems (SHAMAN), New York, 2002.
[39]王胜文.基于软件的故障注入方法研究[D].哈尔滨:哈尔滨工业大学学位论文,2005:28-29.
[40]荣盘祥.复杂系统脆性理论及其理论框架的研究[D].哈尔滨:哈尔滨工程大学学位论文,2006:1-50.
[41]凌复华.突变理论及其应用[M].上海:上海交通大学出版社,1987:1-100.
[42] Zhao L, Tang M K. Dynamic Risk Evaluation of Geological Catastrophe of Tailing Reservoir Based on Catastrophe Theory[C]//Proc. of 4th International Conference on Bioinformatics and Biomedical Engineering. Chengdu: IEEE CS, 2010:1-4.
[43] Oliveira G M, Cechin S L, Weber T S. Dependability Evaluation of Distributed Systems through Partitioning Fault Injection[C]//Proc. of 11th Latin American Test Workshop. Pule del Este: IEEE CS, 2010:1-6.
[44] Schmid M, Ghosh A, Hill F. Techniques for Evaluating the Robustness of Windows NT software[C]//Proc. of the 2000 DARPA InformationSurvivability Conf. and Exposition. Los Alamitos: IEEE Computer Society, 2000:347?360.
[45] Aidemark J, Vinter J , Folkesson P, et al. GOOFI: Generic Object-oriented Fault Injection Tool[C]//Proc. of the Int’l Conf. on Dependable Systems and Networks. Los Alamitos: IEEE Computer Society, 2001: 83?88.
[46] Kao W I, Iyer R K, Tang D. FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX system Behavior under Faults. IEEE Trans. on Software Engineering, 1993, 19(11):1105?1118.
[47] Some R R, Kim W S, Khanoyan G, et al. A Software-implemented Fault Injection Methodology for Design and Validation of System Fault Tolerance[C]//Proc. of the 2001 International Conference on Dependable Systems and Networks. Los Alamitos: IEEE Computer Society, 2001: 501-506.
[48] Some R R, Kim W S, Khanoyan G, et al. Fault Injection Experiment Results in Space Borne Parallel Application Programs[C]//Proc. of IEEE Aerospace Conference. Los Alamitos: IEEE Computer Society, 2002: 2133-2147.
[49]陈锦富,卢炎生,谢晓东.软件错误注入测试技术研究[J].软件学报,2009, 20(6):1425-1433.
[50]彭俊杰.基于软件故障注入的星载系统可靠性评测[D].哈尔滨:哈尔滨工业大学学位论文,2005:20-21.
[51] Li J H, Dai G X, Li H H. Mutation Analysis for Testing Finite State Machines[C]//Proc. of ISECS '09. Second International Symposium on Electronic Commerce and Security. Nanchang: IEEE CS, 2009: 620-624.
[52] Chow T S. Testing Software Design Modelled by Finite State Machines[J]. IEEE Transactions on Software Engineering, 1978, 4(3):178-187.
[53] Fabbri S C P F, Delamaro M E, Maldonado J C, et al. Mutation Analysis Testing for Finite State Machines[C]//Proceedings of the fifth International Symposium on Software Reliability Engineering. Los Alamitos,CA: IEEE Press, 1994: 220-229.
[54] Black P E, Okun V, Yesha Y. Mutation of Model Checker Specification for Test Generation and Evaluation[C]//Proceedings of Mutation 2000. San Jose: California, 2000:14-20.
[55] Schuler D, Dallmeier V, Zeller A. Efficient Mutation Testing by Checking Invariant Violations[C]//Proc. of the eighteenth international symposium onSoftware testing and analysis. New York: ACM, 2009: 69-80.
[56] Delamaro M E, Maldonado J C. Interface Mutation: Assessing Testing Quality at Interprocedural Level[C]//Proceedings of the 19th International Conference of the Chilean Computer Science Society (SCCC’99).Talca, Chile,1999 : 78–86.
[57] Batth S S, Vieira E R, Cavalli A R, et al. Specification of Timed EFSM Fault Models in SDL[C]//Proceedings of the 27th IFIP WG 6.1 International Conference on Formal Techniques for Networked and Distributed Systems (FORTE’07). Tallinn, Estonia: Springer, 2007: 50–65.
[58] Fraser G, Wotawa F. Mutant Minimization for Model-Checker Based Test-Case Generation[C]//Proceedings of the 3rd Workshop on Mutation Analysis (MUTATION’07). Windsor, UK: IEEE Computer Society, 2007: 161–168.
[59] Souza S D R S D, Maldonado J C, Fabbri S C P F, et al. Mutation Testing Applied to Estelle Specifications[J]. Software Quality Control, 1999, 8(4):285-301.
[60] Fabbri S C P F, Maldonado J C, Masiero P C, et al. Mutation Testing Applied to Validate Specifications Based on Petri Nets[C]//Proceedings of the IFIP TC6 8th International Conference on Formal Description Techniques. 1995:329–337.
[61] Jing C, Wang Z, Shi X, et al. Mutation Testing of Protocol Messages Based on Extended TTCN-3[C]//Proceedings of the 22nd International Conference on Advanced Information Networking and Applications (AINA’08). Okinawa, Japan, 2008 :667–674.
[62] Le Traon Y, Mouelhi T, Baudry B. Testing Security Policies: Going Beyond Functional Testing[C]//The 18th IEEE International Symposium on Software Reliability. Sweden: IEEE Computer Society, 2007: 93–102.
[63] Lee S, Bai X, Chen Y. Automatic Mutation Testing and Simulation on OWL-S Specified Web Services[C]//Proceedings of the 41st Annual Simulation Symposium (ANSS’08).Ottawa, Canada, 2008:149–156.
[64] Jia Y, Harman M. An Analysis and Survey of the Development of Mutation Testing[J]. IEEE Transaction on Software Engineering, 2010.
[65] Acree A T, Budd T A, DeMillo R A, et al. Mutation Analysis[R].Georgia :Georgia Institute of Technology, 1979: GIT-ICS-79/08.
[66] DeMillo R A, Lipton R J, Sayward F G. Hints on Test Data Selection: Help for the Practicing Programmer[J]. Computer, 1978, 11(4): 34–41.
[67] Offutt A. The Coupling Effect: Fact or Fiction[C]//Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification. Key west, Florida, United States, 1989:131-140.
[68] Lipton R J, Sayward F G. The Status of Research on Program Mutation[C]//Proceedings of the Workshop on Software Testing and Test Documentation. 1978: 355–373.
[69] Offutt A J. Investigations of the Software Testing Coupling Effect[J]. ACM Transactions on Software Engineering and Methodology, 1992, 1(1):5–20.
[70] Kapoor K. Formal Analysis of Coupling Hypothesis for Logical Faults[J]. Innovations in Systems and Software Engineering, 2006, 2(2):80–87.
[71] AMMANN P, Offutt J. Instruction to Software Testing. Cambridge University Press. 2008, 180-181
[72] King K N, Offutt A J. A Fortran Language System for Mutation-Based Software Testing[J]. Software:Practice and Experience, 1991, 21(7):685-718.
[73] Jia Y, Harman M. MILU: A Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language[C]//Proceedings of the 3rd Testing: Academic and Industrial Conference Practice and Research Techniques (TAIC PART’08). Windsor, UK: IEEE Computer Society, 2008:94–98.
[74] Simao A, Maldonado J C, da Silva Bigonha R. A Transformational Language for Mutant Description[J]. Computer Languages, Systems & Structures, 2009, 35(3): 322–339.
[75] Budd T A, Angluin D. Two Notions of Correctness and Their Relation to Testing[J]. Acta Informatica, 1982,18(1):31–45.
[76] Grun B J M, Schuler D, Zeller A. The Impact of Equivalent Mutants[C]// Proceedings of the 4th International Workshop on Mutation Analysis (MUTATION’09). Denver, Colorado: IEEE Computer Society, 2009:192–199.
[77] Baldwin D, Sayward F G. Heuristics for Determining Equivalence of Program Mutations[R]. Yale University, New Haven, Connecticut, Research Report 276, 1979.
[78] Offutt A J, Craft W M. Using Compiler Optimization Techniques to Detect Equivalent Mutants[J]. Software Testing, Verification and Reliability, 1994, 4(3):131–154.
[79] Offutt A J, Pan J. Detecting Equivalent Mutants and the Feasible Path Problem[C]//Proceedings of the 1996 Annual Conference on Computer Assurance. Gaithersburg, Maryland: IEEE Computer Society Press, 1996: 224–236.
[80] Voas J, McGraw G. Software Fault Injection: Inoculating Programs Against Errors[M]. USA: John Wiley & Sons, 1997: 1-100.
[81] Hierons R M, Harman M, Danicic S. Using Program Slicing to Assist in the Detection of Equivalent Mutants[J]. Software Testing, Verification and Reliability, 1999:9(4):233–262.
[82] Harman M, Hierons R, Danicic S. The Relationship between Program Dependence and Mutation Analysis[C]//Proceedings of the 1st Workshop on Mutation Analysis (MUTATION’00).San Jose: IEEE CS, 2001:5–13.
[83] Adamopoulos K, Harman M, Hierons R M. How to Overcome the Equivalent Mutant Problem and Achieve Tailored Selective Mutation Using Co-evolution[C]//Proceedings of the Genetic and Evolutionary Computation Conference. Seattle, Washington, USA: Springer, 2004: 1338–1349.
[84] Ellims M, Ince D C, Petre M. The Csaw C Mutation Tool: Initial Results[C]// Proceedings of the 3rd Workshop on Mutation Analysis (MUTATION’07).. Windsor, UK: IEEE Computer Society, 2007: 185–192.
[85] Welch B B, Jones K, Hobbs J. Practical Programming in Tcl and Tk[M]. Prentice Hall PTR, 2003:1-50.
[86] Mathur A P. Foundations of Software Testing[M]. Pearson Education, 2008:200-250.
[87] McMillan K L. Symbolic Model Checking[M]. Kluwer Academic Publishers, 1993:1-10.
[88] Clarke E M, Jr., Emerson E A , et al. Automatic Verification of Finite-state Concurrent Systems Using Temporal Logic Specifications[J]. ACM Transactions on Programming Languages and Systems, 1986, 8(2):244-263.
[89] Mealy, George H. A Method for Synthesizing Sequential Circuits[J]. Bell Systems Technical Journal, 1955:1045–1079.
[90] Bryant R E. Symbolic Boolean Manipulation with Ordered Binary-decision diagrams[J]. ACM Computing Surveys, 1992, 24(3): 293-318.
[91] Clarke, Grumberg , Peled. Model Checking[M]. The MIT Press, 1999: 14-14.
[92] Ammann P E, Black P E, Majurski W. Using Model Checking to Generate Tests from Specification[C]//Proceedings of Second International Conference on Formal Engineering Methods. 1998: 46-54.
[93] Hunt G, Brubacher D. Detours: binary interception of Win32 functions[C]//Proceedings of the 3rd conference on USENIX Windows NT Symposium. 1999:14-14.
[94] Martin Hiller. A Software Profiling Methodology for Design and Assessment of Dependable Software[D].Sweden: Chalmers University of Technology Ph.D. Thesis, 2002:50-70.
[95] Detours函数介绍[OL]. http://www.cnblogs.com/flying_bat/archive/ 2008/ 04/ 18/1159996.html.
[96] Barlow R E, Proschan F. Mathematical Theory of Reliability[M]. New York: Wiely, 1965.
[97] Barlow R E, Proschan F. Statistical Theory of Reliability and Life Testing[M]. New York: Holt Rinehart and Winston, 1975.
[98]曹晋华,程侃.可靠性数学引论[M].北京:高等教育出版社, 2006: 136-148.
[99] Han S, Shin K, Rosenberg H. DOCTOR: An Integrated Software Fault Injection Environment for Distributed Real-Time Systems[C]//In Int. Computer Performance and Dependability Symp. Erlangen, Germany, 1995: 204–213.
[100]Baldini A, Benso A, Chiusano S, et al.“BOND”: An interposition agents based fault injector for Windows NT[C]//Proc. of the 2000 IEEE Int’l Symp. on Defect and Fault Tolerance in VLSI Systems. Los Alamitos: IEEE CS, 2000: 387?395.
[101]Sanda P N, Kellington J W, Kudva P. Soft-error resilience of the IBM POWER6 processor[J].IBM Journal of Research and Development. 2008, 52(3):275-284.
[102]何书元.随机过程[M].北京:北京大学出版社,2008:25-31.
[103]Li P, Park H, Gao D, et al. Bridging the Gap between Data-Flow and Control-Flow Analysis for Anomaly Detection[C]//Proc. of 2008 AnnualComputer Security Applications Conference. IEEE CS, 2008:392-401.
[104]Dawson S, Jahanian F, Mitton T. A Software Fault Injection Tool on Real-time Mach[C]//Proceedings of the 16th IEEE Real-Time Systems Symposium. 1995: 130-140.
[105]Stott D T, Floering B, Burke D, et al. Nftape: a Framework for Assessing Dependability in Distributed Systems with Lightweight Fault Injectors[C]//Int. Computer Performance and Dependability Symp. Chicago IL, 2000: 91–100.
[106]Eric Wong W, Yu Qi, Lei Zhao, et al. Effective Fault Localization using Code Coverage[C]//Proc. of the 31st Annual International Computer Software and Applications Conference. Beijing: IEEE CS, 2007: 449-456.
[107]Torsten Lorentzen. Attitude Control and Determination System for DTUsat1[OL].2002.http://microsat.sm.bmstu.ru/e-library/Algorithms/CommonDesign/dtu_torsten.pdf
[108]Felix R. Hoots and Ronald L. Roehrich. Space track report no.3 [OL].1980.http://celestrack.com/NORAD/documentation/spacetrk.pdf.