Experimental Validation of a Risk Assessment Method

详细信息    查看全文

• 作者：Eelco Vriezekolk (15) (17)
  Sandro Etalle (16) (17)
  Roel Wieringa (17)
  
  15. Radiocommunications Agency Netherlands ; Groningen ; Netherlands
  17. University of Twente ; Enschede ; Netherlands
  16. Eindhoven University of Technology ; Eindhoven ; Netherlands
  
• 关键词：Reliability ; Risk assessment ; Expert judgement ; Experiment design ; Telecommunications
• 刊名：Lecture Notes in Computer Science
• 出版年：2015
• 出版时间：2015
• 年：2015
• 卷：9013
• 期：1
• 页码：1-16
• 全文大小：738 KB
• 参考文献：1. Bartha, P.: By Parallel Reasoning. Oxford University Press (2010)
  2. Feather, MS, Cornford, SL (2003) Quantitative risk-based requirements reasoning. Requirements Engineering 8: pp. 248-265 CrossRef
  3. Franqueira, V.N., Tun, T.T., Yu, Y., Wieringa, R., Nuseibeh, B.: Risk and argument: a risk-based argumentation method for practical security. In: 2011 19th IEEE International Requirements Engineering Conference (RE), pp. 239鈥?48. IEEE (2011)
  4. Ghaisas, S., Rose, P., Daneva, M., Sikkel, K.: Generalizing by similarity: Lessons learnt from industrial case studies. In: 1st International Workshop on Conducting Empirical Studies in Industry (CESI), pp. 37鈥?2 (2013)
  5. Gorschek, T, Garre, P, Larsson, S, Wohlin, C (2006) A model for technology transfer in practice. IEEE Software 23: pp. 88-95 CrossRef
  6. Hallgren, KA (2012) Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology 8: pp. 23
  7. Herrmann, A.: Information need of IT risk estimation - qualitative results from experiments. In: Proceedings of the REFSQ 2011 RePriCo Workshop, pp. 72鈥?4 (2011)
  8. Herrmann, A.: REFSQ 2011 live experiment about risk-based requirements prioritization: The influence of wording and metrics. In: Proceedings of REFSQ 2011, pp. 176鈥?94 (2011)
  9. Herrmann, A, Paech, B (2009) Practical challenges of requirements prioritization based on risk estimation. Empirical Software Engineering 14: pp. 644-684 CrossRef
  10. H枚st, M, Regnell, B, Wohlin, C (2000) Using students as subjects鈥揳 comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering 5: pp. 201-214 3/A:1026586415054" target="_blank" title="It opens in new window">CrossRef
  11. IEC: Analysis techniques for system reliability - Procedure for failure mode and effects analysis (FMEA). International Standard 60812:2006 (2006)
  12. ISO: Risk management - principles and guidelines. International Standard 31000 (2009)
  13. Kerckhoffs Institute: The Kerckhoffs masters programme. offs-institute.org/" class="a-plus-plus">http://www.kerchoffs-institute.org/ Last accessed 2014鈥?7-10
  14. Krippendorff, K.: Calculation of alpha over partitions (Private communication)
  15. Krippendorff, K.: Content analysis: an introduction to its methodology. 2nd edn. Sage Publications (2004)
  16. Ojameruaye, B, Bahsoon, R Systematic elaboration of compliance requirements using compliance debt and portfolio theory. In: Salinesi, C, Weerd, I eds. (2014) Requirements Engineering: Foundation for Software Quality. Springer, Heidelberg, pp. 152-167 3-319-05843-6_12" target="_blank" title="It opens in new window">CrossRef
  17. Runeson, P.: Using students as experiment subjects-an analysis on graduate and freshmen student data. In: Proceedings of the 7th International Conference on Empirical Assessment in Software Engineering.-Keele University, UK, pp. 95鈥?02. Citeseer (2003)
  18. Shadish, W., Cook, T., Campbell, D.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company (2002)
  19. Sunstein, C (1993) On analogical reasoning. Harvard Law Review 106: pp. 741-790 307/1341662" target="_blank" title="It opens in new window">CrossRef
  20. Svahnberg, M., Aurum, A., Wohlin, C.: Using students as subjects-an empirical evaluation. In: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 288鈥?90. ACM (2008)
  21. Vriezekolk, E.: Testing reliability of Raster - report of experiment with Kerckhoffs students. Technical report, University of Twente (2014)
  22. Vriezekolk, E., Wieringa, R., Etalle, S.: A new method to assess telecom service availability risks. In: Mendonca, D., Dugdale, J. (eds.) Proceedings of the 8th International Conference on Information Systems for Crisis Response and Management ISCRAM2011 (2011)
  23. Vriezekolk, E., Wieringa, R., Etalle, S.: Design and initial validation of the Raster method for telecom service availability risk assessment. In: Rothkrantz, L., Ristvej, J., Franco, Z. (eds.) Proceedings of the 9th International Conference on Information Systems for Crisis Response and Management ISCRAM2012 (2012)
  24. Vriezekolk, E.: Raster documentation website http://wwwhome.ewi.utwente.nl/~vriezekolk/Raster/
  25. Wieringa, R.: Design Science Methodology for Information Systems and Software Engineering. Springer (2014)
  26. Willner, P.: Methods for assessing the validity of animal models of human psychopathology. In: Boulton, A., Baker, G., Martin-Iverson, M. (eds.) Animal Models in Psychiatry, I, Neuromethods vol. 18, pp. 1鈥?3. Humana Press (1991)
  27. Yin, R.K.: Case study research: design and methods - fourth edition. Sage Publications (2009)
  
• 作者单位：Requirements Engineering: Foundation for Software Quality
• 丛书名：978-3-319-16100-6
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349

文摘

[Context and motivation] It is desirable that requirement engineering methods are reliable, that is, that methods can be repeated with the same results. Risk assessments methods, however, often have low reliability when they identify risk mitigations for a system based on expert judgement. [Question/problem] Our goal is to assess the reliability of an availability risk assessment method for telecominfrastructures, and to identify possibilities for improvement of its reliability. [Principal ideas/results] We propose an experimental validation of reliability, and report on its application. We give a detailed analysis of sources of variation, explain how we controlled them and validated their mitigations, and motivate the statistical procedure used to analyse the outcome. [Contribution] Our results can be used to improve the reliability of risk assessment methods. Our approach to validating reliability can be useful for the assessment of the reliability of other methods.