复杂仿真系统可信度评估研究进展
|
摘要
建模与仿真技术应用广泛,现已成为人们认识和改造现实世界的重要手段,而仿真系统可信与否是仿真用户十分关注的问题.随着仿真对象愈发复杂以及用户对仿真应用要求越来越高,仿真系统呈现出多种复杂特性,以致于其可信度评估面临诸多挑战.本文首先总结了国内外仿真可信度评估发展历程,并分析了复杂仿真系统的特点及可信度评估面临的问题;然后,在此基础上给出了复杂仿真系统可信度评估解决方案及应对策略;最后总结了复杂仿真系统可信度评估所面临的挑战与机遇,指出了未来研究方向.
Modeling and simulation have been used extensively, and they have become an important means to recognize and remold the real world. Whether the simulation system is credible has been a concern of simulation users. Various complexity characteristics of the simulation systems are presented with increasingly complex simulation objects and higher application requirements, creating some challenges in credibility assessment. First,the development process of the simulation credibility assessment is summarized. Next, the characteristics of complex simulation systems and the problems of the credibility assessment are analyzed, and some solutions and countermeasures for the assessment problems are provided. Finally, the challenges and opportunities of the credibility assessment for complex simulation systems are summarized, and the corresponding future research directions are indicated.
Modeling and simulation have been used extensively, and they have become an important means to recognize and remold the real world. Whether the simulation system is credible has been a concern of simulation users. Various complexity characteristics of the simulation systems are presented with increasingly complex simulation objects and higher application requirements, creating some challenges in credibility assessment. First,the development process of the simulation credibility assessment is summarized. Next, the characteristics of complex simulation systems and the problems of the credibility assessment are analyzed, and some solutions and countermeasures for the assessment problems are provided. Finally, the challenges and opportunities of the credibility assessment for complex simulation systems are summarized, and the corresponding future research directions are indicated.
引文
1 Wang Z C.Development and formation of simulation science.J Syst Simul,2005,06:1279–1281[王子才.仿真科学的发展及形成.系统仿真学报,2005,06:1279–1281]
2 Yang M,Zhang B,Wang Z C.The analysis of modeling and simulation development direction.J Syst Simul,2004,16 :1901–1904[杨明,张冰,王子才.建模与仿真技术发展趋势分析.系统仿真学报,2004,16:1901–1904]
3 Robinson S.Simulation:the Practice of Model Development and Use.London:Palgrave Macmillan Press,2014.18–21
4 Wang Z C,Zhang B,Yang M.Verification,validation and accreditation(VV&A)for simulation system:current status and future.J Syst Simul,1999,11:321–326[王子才,张冰,杨明.仿真系统的校核、验证与验收(VV&A):现状与未来.系统仿真学报,1999,11:321–326]
5 Qian X C.Research on simulation model validation and calibration methods under uncertainty.Dissertation for Ph.D.Degree.Harbin:Harbin Institute of Technology,2016[钱晓超.考虑不确定性影响的仿真模型验证及校准方法研究.博士学位论文.哈尔滨:哈尔滨工业大学,2016]
6 Yang M,Zhang B,Ma P,et al.Five key issues of the development of simulation systems VV&A.J Syst Simul,2003,15 :1502–1506[杨明,张冰,马萍,等.仿真系统VV&A发展的五大关键问题.系统仿真学报,2003,15:1502–1506]
7 Sargent R G.Verification and validation of simulation models.J Simul,2013,7:12–24
8 Distributed Interactive Simulation Committee of the IEEE Computer Society.IEEE Recommended Practice for Distributed Interactive Simulation—Verification,Validation,and Accreditation.Version 1278.4,1997
9 Conway R W,Johnson B M,Maxwell W L.Some problems of digital systems simulation.Manage Sci,1959,6:92–110
10 Fishman G S,Kiviat P J.The analysis of simulation generated time series.Manage Sci,1967,3:525–557
11 Hermann C F.Validation problems in games and simulations with special reference to models of international politics.Syst Res,1967,12:216–231
12 Schruben L W.Establishing the credibility of simulations.Simulation,1980,34:101–105
13 Teorey T J.Validation criteria for computer system simulations.Simuletter,1975,6:9–20
14 Balci O,Sargent R G.Some examples of simulation model validation using hypothesis testing.In:Proceedings of the 14 th Conference on Winter Simulation,San Diego,1982.621–629
15 Balci O,Sargent R G.Validation of simulation models via simultaneous confidence intervals.Am J Math Manage Sci,1984,4:375–406
16 Kheir N A,Holmes W M.On validating simulation models of missile systems.Simulation,1978,30:117–128
17 Tytula T P.A Method for Validating Missile System Simulation Models.Army Missile R&D Command,Redstone Arsenal,1978
18 Montgomery D C,Conard R G.Comparison of simulation and flight-test data for missile systems.Simulation,1980,34 :63–72
19 Balci O.Validation,verification,and testing techniques throughout the life cycle of a simulation study.Ann Oper Res,1994,53:121–173
20 Kleijnen J P C.Verification and validation of simulation models.Eur J Oper Res,1995,82:145–162
21 Defense Modeling and Simulation Office.Verification,Validation and Accreditation Recommended Practice Guide BUILD 1.1996
22 Defense Modeling and Simulation Office.Verification,Validation and Accreditation Recommended Practice Guide BUILD 2.2000
23 Simulation Interoperability Standards Committee of the IEEE Computer Society.IEEE Recommended Practice for Verification,Validation,and Accreditation of A Federation:an Overlay to the High Level Architecture Federation Development and Execution Process.Version 1516.4,2007
24 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.1:Introduction and Overview Version 1,2012
25 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.2:Implementation Guide Version 2,2013
26 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.3:Reference Manual Version 3,2013
27 Mullins J,Ling Y,Mahadevan S,et al.Separation of aleatory and epistemic uncertainty in probabilistic model validation.Reliab Eng Syst Safe,2016,147:49–59
28 Zhao L F,Lu Z Z,Yun W Y,et al.Validation metric based on Mahalanobis distance for models with multiple correlated responses.Reliab Eng Syst Safe,2017,159:80–89
29 Kwag S,Gupta A,Dinh N.Probabilistic risk assessment based model validation method using Bayesian network.Reliab Eng Syst Safe,2018,169:380–393
30 Ao D,Hu Z,Mahadevan S.Design of validation experiments for life prediction models.Reliab Eng Syst Safe,2017,165 :22–33
31 Wu D Q,Lu Z Z,Wang Y P,et al.Model validation and calibration based on component functions of model output.Reliab Eng Syst Safe,2015,140:59–70
32 Li Y L,Wang X J,Wang C,et al.Non-probabilistic Bayesian update method for model validation.Appl Math Model,2018,58:388–403
33 Jiang X M,Yuan Y,Mahadevan S,et al.An investigation of Bayesian inference approach to model validation with non-normal data.J Stat Comput Simul,2013,83:1829–1851
34 Ling Y,Mahadevan S.Quantitative model validation techniques:new insights.Reliab Eng Syst Safe,2013,111:217 –231
35 Sankararaman S,Mahadevan S.Integration of model verification,validation,and calibration for uncertainty quantification in engineering systems.Reliab Eng Syst Safe,2015,138:194–209
36 Atkinson A D,Hill R R,Pignatiello J J,et al.Wavelet ANOVA approach to model validation.Simul Model Pract Theory,2017,78:18–27
37 Sarin H,Kokkolaras M,Hulbert G,et al.A comprehensive metric for comparing time histories in validation of simulation models with emphasis on vehicle safety applications.In:Proceedings of the ASME 2008 International Design Engineering Technical Conferences&Computers and Information in Engineering Conference,New York,2009
38 Poropudas J,Virtanen K.Game theoretic validation of air combat simulation models.In:Proceedings of the 2009IEEE International Conference on Systems,Man,and Cybernetics,San Antonio,2009.3243–3250
39 Lemmers A,Roza M,Voogd J,et al.V&V study of an F-16 familiarization training simulator.In:Fall Simulation Interoperability Workshop.USA:SISO,2013.98–108
40 Dhananjayan A,Seow K T.A formal transparency framework for validation of real-time discrete-event control requirements modeled by timed transition graphs.IEEE Trans Hum-Mach Syst,2015,45:350–361
41 Guerini M,Moneta A.A method for agent-based models validation.J Econ Dyn Control,2017,82:125–141
42 Jebeile J,Barberousse A.Empirical agreement in model validation.Stud Hist Philos Sci Part A,2016,56:168–174
43 Michopoulos J,Lambrakos S.On the fundamental tautology of validating data-driven models and simulations.In:Proceedings of International Conference on Computational Science,Atlanta,2005.738–745
44 Szabo C,Teo Y M.On validation of semantic composability in data-driven simulation.In:Proceedings of IEEE Workshop on Principles of Advanced and Distributed Simulation,Atlanta,2010.73–80
45 Lamperti F.An information theoretic criterion for empirical validation of simulation models.Economet Stat,2018,5:83 –106
46 Guerini M,Moneta A.A method for agent-based models validation.J Econ Dyn Control,2017,82:125–141
47 Wei H L,Li Z W.Grey relational analysis and its applieation to the validation of computer simulation models for missile systems.Syst Eng Electron,1997,2:55–61[魏华梁,李钟武.灰色关联分析及其在导弹系统仿真模型验证中的应用.系统工程与电子技术,1997,2:55–61]
48 Li W,Jiao S,Lu L Y,et al.Validation and selection of simulation model based on the feature differences.Act Autom Sin,2014,40:2134–2144[李伟,焦松,陆凌云,等.基于特征差异的仿真模型验证及选择方法.自动化学报,2014,40 :2134–2144]
49 Li L,Lu Z Z.A new method for model validation with multivariate output.Reliab Eng Syst Safe,2018,168:579–592
50 Wu Y J,Wang J M,Yang W G.Approach of credibility evaluation for testing system with small samples.J Beijing Univ Aeronaut Astronaut,2016,42:1911–1917[吴云洁,王建敏,杨文光.基于小样本的试验系统可信度评估方法.北京航空航天大学学报,2016,42:1911–1917]
51 Zheng K,Hu J,Zhan Z F,et al.Multivariate responses analysis for model validation of dynamic systems.J Shanghai Jiao Tong Univ,2015,49:191–195[郑凯,胡洁,詹振飞,等.动态系统模型验证的多元响应分析.上海交通大学学报,2015,49:191–195]
52 Wang H Y.Research on multivariate simulation result validation methods under uncertainty.Dissertation for Master Degree.Harbin:Harbin Institute of Technology,2016[王海英.多元不确定仿真结果验证方法研究.硕士学位论文.哈尔滨:哈尔滨工业大学,2016]
53 Min F Y,Yang M,Wang Z C.Knowledge-based method for the validation of complex simulation models.Simul Model Pract Theory,2010,18:500–515
54 Jiao S,Li W,Yang M.Validation method of simulation model based on triangular fuzzy number.J Cent South Univ(Sci Technol),2014,45:124–131[焦松,李伟,杨明.基于三角模糊数的仿真模型验证方法.中南大学学报(自然科学版),2014,45:124–131]
55 Huang Z J,Chen B,Ou Y H.Model checking continuous time Markov process based on timed schedulers.J Guangxi Univ Sci Technol,2014,25:59–63[黄镇谨,陈波,欧阳浩.基于时间策略的连续时间Markov过程验证.广西科技大学学报,2014,25:59–63]
56 Wang S,Wu D H,Qu L,et al.Study on credibility evaluation framework of complex simulation systems.Comput Simul,2012,29:116–122[王石,伍丁红,瞿亮,等.复杂仿真系统可信度评估框架研究.计算机仿真,2012,29:116 –122]
57 Niu S,Lin S L,Li W,et al.Model validation method for discrete event simulation.J Syst Simul,2017,29:1984–1990[牛帅,林圣琳,李伟,等.一种离散事件仿真模型验证方法.系统仿真学报,2017,29:1984–1990]
58 Xia W,Yao Y P,Mu X D,et al.Parallel model checking for discrete event simulation models based on event graphs.J Softw,2012,23:1429–1443[夏薇,姚益平,慕晓冬,等.基于事件图的离散事件仿真模型并行检验方法.软件学报,2012,23:1429–1443]
59 Fang K,Zhou Y C,Zhao E J.Discussion for the factor space of simulation model validation.Syst Eng Electron,2017,39:2592–2602[方可,周玉臣,赵恩娇.关于仿真模型验证指标体系的探讨.系统工程与电子技术,2017,39:2592–2602]
60 Zhao H Y.Analysis on credibility of radar seeker hardware-in-the-loop simulation system.Ship Electron Eng,2015,35 :90–92[赵红云.雷达制导半实物仿真系统可信度分析.舰船电子工程,2015,35:90–92]
61 Lin S L,Li W,Ma P,et al.A new credibility assessment framework for training simulators.In:Proceedings of the8 th International Conference on Computer Modeling and Simulation(ICCMS),Canberra,2017.182–186
62 Zhang Z.Research on evaluation method for credibility of simulation system.Dissertation for Ph.D.Degree.Harbin:Harbin Institute of Technology,2014[张忠.仿真系统可信度评估方法研究.博士学位论文.哈尔滨工业大学,2014]
63 Liu F,Ma P,Yang M,et al.Research on credibility quantification of complex simulation systems.J Harbin Inst Technol.2007,39:1–3[刘飞,马萍,杨明,等.复杂仿真系统可信度量化研究.哈尔滨工业大学学报,2007,39:1–3]
64 Fang K,Zhou Y C,Zhao K B.Validation method for simulation models with iteration operation.Syst Eng Electron,2017,39:445–450[方可,周玉臣,赵开斌.含有迭代运算的仿真模型验证方法.系统工程与电子技术,2017,39:445 –450]
65 Fang K,Yang M,Wang Z C.The HITVICE VV&A environment.In:Proceedings of the Winter Simulation Conference,Orlando,2005.1220–1227
66 Ju R S,Yang M,Huang K D,et al.Summary of service oriented modeling and simulation.Syst Eng Electron,2013,35:1539–1546[鞠儒生,杨妹,黄柯棣,等.面向服务的建模与仿真技术综述.系统工程与电子技术,2013,35:1539–1546]
67 Li B H,Chai X D,Hou B C,et al.Networked modeling&simulation platform based on concept of cloud computing—cloud simulation platform.J Syst Simul,2009,21:5292–5299[李伯虎,柴旭东,候宝存,等.一种基于云计算理念的网络化建模与仿真平台——云仿真平台.系统仿真学报,2009,21:5292–5299]
2 Yang M,Zhang B,Wang Z C.The analysis of modeling and simulation development direction.J Syst Simul,2004,16 :1901–1904[杨明,张冰,王子才.建模与仿真技术发展趋势分析.系统仿真学报,2004,16:1901–1904]
3 Robinson S.Simulation:the Practice of Model Development and Use.London:Palgrave Macmillan Press,2014.18–21
4 Wang Z C,Zhang B,Yang M.Verification,validation and accreditation(VV&A)for simulation system:current status and future.J Syst Simul,1999,11:321–326[王子才,张冰,杨明.仿真系统的校核、验证与验收(VV&A):现状与未来.系统仿真学报,1999,11:321–326]
5 Qian X C.Research on simulation model validation and calibration methods under uncertainty.Dissertation for Ph.D.Degree.Harbin:Harbin Institute of Technology,2016[钱晓超.考虑不确定性影响的仿真模型验证及校准方法研究.博士学位论文.哈尔滨:哈尔滨工业大学,2016]
6 Yang M,Zhang B,Ma P,et al.Five key issues of the development of simulation systems VV&A.J Syst Simul,2003,15 :1502–1506[杨明,张冰,马萍,等.仿真系统VV&A发展的五大关键问题.系统仿真学报,2003,15:1502–1506]
7 Sargent R G.Verification and validation of simulation models.J Simul,2013,7:12–24
8 Distributed Interactive Simulation Committee of the IEEE Computer Society.IEEE Recommended Practice for Distributed Interactive Simulation—Verification,Validation,and Accreditation.Version 1278.4,1997
9 Conway R W,Johnson B M,Maxwell W L.Some problems of digital systems simulation.Manage Sci,1959,6:92–110
10 Fishman G S,Kiviat P J.The analysis of simulation generated time series.Manage Sci,1967,3:525–557
11 Hermann C F.Validation problems in games and simulations with special reference to models of international politics.Syst Res,1967,12:216–231
12 Schruben L W.Establishing the credibility of simulations.Simulation,1980,34:101–105
13 Teorey T J.Validation criteria for computer system simulations.Simuletter,1975,6:9–20
14 Balci O,Sargent R G.Some examples of simulation model validation using hypothesis testing.In:Proceedings of the 14 th Conference on Winter Simulation,San Diego,1982.621–629
15 Balci O,Sargent R G.Validation of simulation models via simultaneous confidence intervals.Am J Math Manage Sci,1984,4:375–406
16 Kheir N A,Holmes W M.On validating simulation models of missile systems.Simulation,1978,30:117–128
17 Tytula T P.A Method for Validating Missile System Simulation Models.Army Missile R&D Command,Redstone Arsenal,1978
18 Montgomery D C,Conard R G.Comparison of simulation and flight-test data for missile systems.Simulation,1980,34 :63–72
19 Balci O.Validation,verification,and testing techniques throughout the life cycle of a simulation study.Ann Oper Res,1994,53:121–173
20 Kleijnen J P C.Verification and validation of simulation models.Eur J Oper Res,1995,82:145–162
21 Defense Modeling and Simulation Office.Verification,Validation and Accreditation Recommended Practice Guide BUILD 1.1996
22 Defense Modeling and Simulation Office.Verification,Validation and Accreditation Recommended Practice Guide BUILD 2.2000
23 Simulation Interoperability Standards Committee of the IEEE Computer Society.IEEE Recommended Practice for Verification,Validation,and Accreditation of A Federation:an Overlay to the High Level Architecture Federation Development and Execution Process.Version 1516.4,2007
24 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.1:Introduction and Overview Version 1,2012
25 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.2:Implementation Guide Version 2,2013
26 Simulation Interoperability Standards Organization.Reference for Generic Methodology for Verification and Validation(GM-VV)to Support Acceptance of Models,Simulations and Data.Vol.3:Reference Manual Version 3,2013
27 Mullins J,Ling Y,Mahadevan S,et al.Separation of aleatory and epistemic uncertainty in probabilistic model validation.Reliab Eng Syst Safe,2016,147:49–59
28 Zhao L F,Lu Z Z,Yun W Y,et al.Validation metric based on Mahalanobis distance for models with multiple correlated responses.Reliab Eng Syst Safe,2017,159:80–89
29 Kwag S,Gupta A,Dinh N.Probabilistic risk assessment based model validation method using Bayesian network.Reliab Eng Syst Safe,2018,169:380–393
30 Ao D,Hu Z,Mahadevan S.Design of validation experiments for life prediction models.Reliab Eng Syst Safe,2017,165 :22–33
31 Wu D Q,Lu Z Z,Wang Y P,et al.Model validation and calibration based on component functions of model output.Reliab Eng Syst Safe,2015,140:59–70
32 Li Y L,Wang X J,Wang C,et al.Non-probabilistic Bayesian update method for model validation.Appl Math Model,2018,58:388–403
33 Jiang X M,Yuan Y,Mahadevan S,et al.An investigation of Bayesian inference approach to model validation with non-normal data.J Stat Comput Simul,2013,83:1829–1851
34 Ling Y,Mahadevan S.Quantitative model validation techniques:new insights.Reliab Eng Syst Safe,2013,111:217 –231
35 Sankararaman S,Mahadevan S.Integration of model verification,validation,and calibration for uncertainty quantification in engineering systems.Reliab Eng Syst Safe,2015,138:194–209
36 Atkinson A D,Hill R R,Pignatiello J J,et al.Wavelet ANOVA approach to model validation.Simul Model Pract Theory,2017,78:18–27
37 Sarin H,Kokkolaras M,Hulbert G,et al.A comprehensive metric for comparing time histories in validation of simulation models with emphasis on vehicle safety applications.In:Proceedings of the ASME 2008 International Design Engineering Technical Conferences&Computers and Information in Engineering Conference,New York,2009
38 Poropudas J,Virtanen K.Game theoretic validation of air combat simulation models.In:Proceedings of the 2009IEEE International Conference on Systems,Man,and Cybernetics,San Antonio,2009.3243–3250
39 Lemmers A,Roza M,Voogd J,et al.V&V study of an F-16 familiarization training simulator.In:Fall Simulation Interoperability Workshop.USA:SISO,2013.98–108
40 Dhananjayan A,Seow K T.A formal transparency framework for validation of real-time discrete-event control requirements modeled by timed transition graphs.IEEE Trans Hum-Mach Syst,2015,45:350–361
41 Guerini M,Moneta A.A method for agent-based models validation.J Econ Dyn Control,2017,82:125–141
42 Jebeile J,Barberousse A.Empirical agreement in model validation.Stud Hist Philos Sci Part A,2016,56:168–174
43 Michopoulos J,Lambrakos S.On the fundamental tautology of validating data-driven models and simulations.In:Proceedings of International Conference on Computational Science,Atlanta,2005.738–745
44 Szabo C,Teo Y M.On validation of semantic composability in data-driven simulation.In:Proceedings of IEEE Workshop on Principles of Advanced and Distributed Simulation,Atlanta,2010.73–80
45 Lamperti F.An information theoretic criterion for empirical validation of simulation models.Economet Stat,2018,5:83 –106
46 Guerini M,Moneta A.A method for agent-based models validation.J Econ Dyn Control,2017,82:125–141
47 Wei H L,Li Z W.Grey relational analysis and its applieation to the validation of computer simulation models for missile systems.Syst Eng Electron,1997,2:55–61[魏华梁,李钟武.灰色关联分析及其在导弹系统仿真模型验证中的应用.系统工程与电子技术,1997,2:55–61]
48 Li W,Jiao S,Lu L Y,et al.Validation and selection of simulation model based on the feature differences.Act Autom Sin,2014,40:2134–2144[李伟,焦松,陆凌云,等.基于特征差异的仿真模型验证及选择方法.自动化学报,2014,40 :2134–2144]
49 Li L,Lu Z Z.A new method for model validation with multivariate output.Reliab Eng Syst Safe,2018,168:579–592
50 Wu Y J,Wang J M,Yang W G.Approach of credibility evaluation for testing system with small samples.J Beijing Univ Aeronaut Astronaut,2016,42:1911–1917[吴云洁,王建敏,杨文光.基于小样本的试验系统可信度评估方法.北京航空航天大学学报,2016,42:1911–1917]
51 Zheng K,Hu J,Zhan Z F,et al.Multivariate responses analysis for model validation of dynamic systems.J Shanghai Jiao Tong Univ,2015,49:191–195[郑凯,胡洁,詹振飞,等.动态系统模型验证的多元响应分析.上海交通大学学报,2015,49:191–195]
52 Wang H Y.Research on multivariate simulation result validation methods under uncertainty.Dissertation for Master Degree.Harbin:Harbin Institute of Technology,2016[王海英.多元不确定仿真结果验证方法研究.硕士学位论文.哈尔滨:哈尔滨工业大学,2016]
53 Min F Y,Yang M,Wang Z C.Knowledge-based method for the validation of complex simulation models.Simul Model Pract Theory,2010,18:500–515
54 Jiao S,Li W,Yang M.Validation method of simulation model based on triangular fuzzy number.J Cent South Univ(Sci Technol),2014,45:124–131[焦松,李伟,杨明.基于三角模糊数的仿真模型验证方法.中南大学学报(自然科学版),2014,45:124–131]
55 Huang Z J,Chen B,Ou Y H.Model checking continuous time Markov process based on timed schedulers.J Guangxi Univ Sci Technol,2014,25:59–63[黄镇谨,陈波,欧阳浩.基于时间策略的连续时间Markov过程验证.广西科技大学学报,2014,25:59–63]
56 Wang S,Wu D H,Qu L,et al.Study on credibility evaluation framework of complex simulation systems.Comput Simul,2012,29:116–122[王石,伍丁红,瞿亮,等.复杂仿真系统可信度评估框架研究.计算机仿真,2012,29:116 –122]
57 Niu S,Lin S L,Li W,et al.Model validation method for discrete event simulation.J Syst Simul,2017,29:1984–1990[牛帅,林圣琳,李伟,等.一种离散事件仿真模型验证方法.系统仿真学报,2017,29:1984–1990]
58 Xia W,Yao Y P,Mu X D,et al.Parallel model checking for discrete event simulation models based on event graphs.J Softw,2012,23:1429–1443[夏薇,姚益平,慕晓冬,等.基于事件图的离散事件仿真模型并行检验方法.软件学报,2012,23:1429–1443]
59 Fang K,Zhou Y C,Zhao E J.Discussion for the factor space of simulation model validation.Syst Eng Electron,2017,39:2592–2602[方可,周玉臣,赵恩娇.关于仿真模型验证指标体系的探讨.系统工程与电子技术,2017,39:2592–2602]
60 Zhao H Y.Analysis on credibility of radar seeker hardware-in-the-loop simulation system.Ship Electron Eng,2015,35 :90–92[赵红云.雷达制导半实物仿真系统可信度分析.舰船电子工程,2015,35:90–92]
61 Lin S L,Li W,Ma P,et al.A new credibility assessment framework for training simulators.In:Proceedings of the8 th International Conference on Computer Modeling and Simulation(ICCMS),Canberra,2017.182–186
62 Zhang Z.Research on evaluation method for credibility of simulation system.Dissertation for Ph.D.Degree.Harbin:Harbin Institute of Technology,2014[张忠.仿真系统可信度评估方法研究.博士学位论文.哈尔滨工业大学,2014]
63 Liu F,Ma P,Yang M,et al.Research on credibility quantification of complex simulation systems.J Harbin Inst Technol.2007,39:1–3[刘飞,马萍,杨明,等.复杂仿真系统可信度量化研究.哈尔滨工业大学学报,2007,39:1–3]
64 Fang K,Zhou Y C,Zhao K B.Validation method for simulation models with iteration operation.Syst Eng Electron,2017,39:445–450[方可,周玉臣,赵开斌.含有迭代运算的仿真模型验证方法.系统工程与电子技术,2017,39:445 –450]
65 Fang K,Yang M,Wang Z C.The HITVICE VV&A environment.In:Proceedings of the Winter Simulation Conference,Orlando,2005.1220–1227
66 Ju R S,Yang M,Huang K D,et al.Summary of service oriented modeling and simulation.Syst Eng Electron,2013,35:1539–1546[鞠儒生,杨妹,黄柯棣,等.面向服务的建模与仿真技术综述.系统工程与电子技术,2013,35:1539–1546]
67 Li B H,Chai X D,Hou B C,et al.Networked modeling&simulation platform based on concept of cloud computing—cloud simulation platform.J Syst Simul,2009,21:5292–5299[李伯虎,柴旭东,候宝存,等.一种基于云计算理念的网络化建模与仿真平台——云仿真平台.系统仿真学报,2009,21:5292–5299]