“GIOSAT- a tool to assess CanMEDS competencies during simulated crises

详细信息    查看全文

• 作者：Victor M. Neira MD (1)
  M. Dylan Bould MBChB (1)
  Amy Nakajima MD (2)
  Sylvain Boet MD (3)
  Nicholas Barrowman PhD (4)
  Philipp Mossdorf MD ; PhD (1)
  Devin Sydor MD (5)
  Amy Roeske MD (1)
  Stephen Noseworthy MD (6)
  Viren Naik MD (7)
  Dermot Doherty MD (1)
  Hilary Writer MD (8)
  Stanley J. Hamstra PhD (9)
  
• 刊名：Canadian Journal of Anesthesia/Journal canadien d'anesth篓娄sie
• 出版年：2013
• 出版时间：March 2013
• 年：2013
• 卷：60
• 期：3
• 页码：280-289
• 全文大小：446KB
• 参考文献：1. / The Royal College of Physicians and Surgeons of Canada. CanMEDS 2005 Framework. CanMEDS: better standards, better physicians, better care. Available from URL: http://www.rcpsc.medical.org (accessed November 2012).
  2. / Sherbino J, / Frank JR, / Flynn L, / Snell L. “Intrinsic roles-rather than “armour- renaming the “non-medical expert roles-of the CanMEDS framework to match their intent. Adv Health Sci Educ Theory Pract 2011; 16: 695-7. CrossRef
  3. / Kohn LT, / Corrigan JM, / Donaldson MS. To Err is Human: Building a Safer Health Care System. Washington, DC: National Academy Press; 1999 .
  4. / Canadian Institute for Health Information. Health Care in Canada, 2004. Available from URL: www.cihi.ca (accessed November 2012).
  5. / McIlvaine WB. Human error and its impact on anesthesiology. Semin Anesth 2006; 25: 172-9.
  6. / Jimenez N, / Posner KL, / Cheney FW, / Caplan RA, / Lee LA, / Domino KB. An update on pediatric anesthesia liability: a closed claims analysis. Anesth Analg 2007; 104: 147-53. CrossRef
  7. / Gaba DM, / Howard SK, / Fish KJ, / Smith BE, / Sowb YA. Simulation-based training in anesthesia crisis resource management (ACRM): a decade of experience. Simulation & Gaming 2001; 32: 175-93. CrossRef
  8. / Fletcher G, / Flin R, / McGeorge P, / Glavin R, / Maran N, / Patey R. Anesthetists-Non-Technical Skills (ANTS): evaluation of a behavioral marker system. Br J Anaesth 2003; 90: 580-8. CrossRef
  9. / Flin R, O’Connor P, Crichton M. Safety at the Sharp End - A Guide to Non-Technical Skills. Farnham: England, Ashgate Publishing Limited; 2008: 1-16.
  10. / Nestel D, / Walker K, / Simon R, / Aggarwal R, / Andreatta P. Nontechnical skills: an inaccurate and unhelpful descriptor? Simul Healthc 2011; 6: 2-3. CrossRef
  11. / Bandiera G, / Sherbino J, / Frank JR. The CanMEDS Assessment Tools Handbook. An Introductory Guide to Assessment Methods for the CanMEDS Competencies. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2006 .
  12. / Hawkins RE, / Boulet JR. Direct observation: standardized patients. In: Holmboe ES, Hawkins RE, editors. Practical Guide to the Evaluation of Clinical Competence. Philadelphia: Mosby Elsevier; 2008. p. 102-18.
  13. / Savoldelli GL, / Nail VN, / Joo HS, / et al. Evaluation of patient simulator performance as an adjunct to the oral examination for senior anesthesia residents. Anesthesiology 2006; 104: 475-81. CrossRef
  14. / Kim J, / Neilipovitz D, / Cardinal P, / Chiu M, / Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med 2006; 34: 2167-74. CrossRef
  15. / Brett-Fleegler MB, / Vinci RJ, / Weiner DL, / Harris SK, / Shih MC, / Kleinman ME. A simulator-based tool that assesses pediatric resident resuscitation competency. Pediatrics 2008; 121: e597-603. CrossRef
  16. / Morgan PJ, / Cleave-Hogg D, / DeSousa S, / Tarshis J. High-fidelity patient simulation: validation of performance checklists. Br J Anaesth 2004; 92: 388-92. CrossRef
  17. / Murray DJ, / Boulet JR, / Kras JF, / Woodhouse JA, / Cox T, / McAllister JD. Acute care skills in anesthesia practice: a simulation-based resident performance assessment. Anesthesiology 2004; 101: 1084-95. CrossRef
  18. / Ringsted C, Hodges B, Scherpbier A. ‘The research compass- an introduction to research in medical education: AMEE Guide No. 56. Med Teach 2011; 33: 695-709.
  19. / Hamstra SJ. Key note address: the focus on competencies and individual learner assessment as emerging themes in medical education research. Acad Emerg Med 2012; 19: 1336-43.
  20. / Graham J, / Hocking G, / Giles E. Anaesthesia non-technical skills: can anaesthetists be trained to reliably use this behavioural marker system in 1?day? Br J Anaesth 2010; 104: 440-5. CrossRef
  21. / Yule S, / Flin R, / Maran N, / Rowley D, / Youngson G, / Paterson-Brown S. Surgeons-non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 2008; 32: 548-56. CrossRef
  22. / Jankouskas T, / Bush MC, / Murray B, / et al. Crisis resource management: evaluating outcomes of a multidisciplinary team. Simul Healthc 2007; 2: 96-101. CrossRef
  23. / Guise JM, / Deering SH, / Kanki BG, / et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc 2008; 3: 217-23. CrossRef
  24. / Malec JF, / Torsher LC, / Dunn WF, / et al. The Mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc 2007; 2: 4-10. CrossRef
  25. / Boulet JR, / Murray D, / Kras J, / Woodhouse J, / McAllister J, / Ziv A. Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology 2003; 99: 1270-80. CrossRef
  26. / Chopra V, / Gesink BJ, / de Jong J, / Bovill JG, / Spierdijk J, / Brand R. Does training on an anaesthesia simulator lead to improvement in performance? Br J Anaesth 1994; 73: 293-7. CrossRef
  27. / Scavone BM, / Sproviero MT, / McCarthy RJ, / et al. Development of an objective scoring system for measurement of resident performance on the human patient simulator. Anesthesiology 2006; 105: 260-6. CrossRef
  28. / Devitt JH, / Kurrek MM, / Cohen MM, / Cleave-Hogg D. The validity of performance assessments using simulation. Anesthesiology 2001; 95: 36-42. CrossRef
  29. / Murray DJ, / Boulet JR, / Avidan M, / et al. Performance of residents and anesthesiologists in a simulation-based skill assessment. Anesthesiology 2007; 107: 705-13. CrossRef
  30. / Murray DJ, / Boulet JR, / Kras JF, / McAllister JD, / Cox TE. A simulation-based acute skills performance assessment for anesthesia training. Anesth Analg 2005; 101: 1127-34. CrossRef
  31. / Gaba DM, / Howard SK, / Flanagan B, / Smith BE, / Fish KJ, / Botney R. Assessment of clinical performance during simulated crisis using both technical and behavioral ratings. Anesthesiology 1998; 89: 8-18. CrossRef
  32. / Kane M, / Crooks T, / Cohen A. Validating measures of performance. Educational Measurement: Issues and Practice 1999; 18: 5-17. CrossRef
  33. / Jones J, / Hunter D. Consensus methods for medical and health services research. BMJ 1995; 311: 376-80. CrossRef
  34. / Bhananker SM, / Ramamoorthy C, / Geiduscheck JM, / et al. Anesthesia-related cardiac arrest in children: update from the Pediatric Perioperative Cardiac Arrest Registry. Anesth Analg 2007; 105: 344-50. CrossRef
  35. / Landis JR, / Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-74. CrossRef
  36. / Boet S, / Bould MD, / Bruppacher HR, / Desjardins F, / Chandra DB, / Naik VN. Looking in the mirror: self-debriefing versus instructor debriefing for simulated crises. Crit Care Med 2011; 39: 1377-81. CrossRef
  37. / Cohen J. Statistical Power Analysis for the Behavioral Sciences, Second Edition. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc., Publishers; 1988.
  38. / American Heart Association. Part 4: Advanced life support. Circulation 2005; 112: III25-54.
  39. / Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat 1979; 6: 65-70.
  40. / Glavin RJ. Skills, training and education. Simul Healthc 2011; 6: 4-7.
  41. / Lynch DC, / Surdyk PM, / Eiser AR. Assessing professionalism: a review of the literature. Med Teach 2004; 26: 366-73. CrossRef
  42. / Bahaziq W, / Crosby E. Physician professional behaviour affects outcomes: a framework for teaching professionalism during anesthesia residency. Can J Anesth 2011; 58: 1039-50. CrossRef
  43. / Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006; 119: 166.e7-16.
  
• 作者单位：Victor M. Neira MD (1)
  M. Dylan Bould MBChB (1)
  Amy Nakajima MD (2)
  Sylvain Boet MD (3)
  Nicholas Barrowman PhD (4)
  Philipp Mossdorf MD, PhD (1)
  Devin Sydor MD (5)
  Amy Roeske MD (1)
  Stephen Noseworthy MD (6)
  Viren Naik MD (7)
  Dermot Doherty MD (1)
  Hilary Writer MD (8)
  Stanley J. Hamstra PhD (9)
  
  1. Department of Anesthesia, Children’s Hospital of Eastern Ontario, University of Ottawa, 401 Smyth Road, Ottawa, ON, K1G 6W3, Canada
  2. Department of Obstetrics and Gynaecology, The Ottawa Hospital, University of Ottawa, Ottawa, ON, Canada
  3. Department of Anesthesia, The Ottawa Hospital, University of Ottawa, Ottawa, ON, Canada
  4. Research Institute, Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada
  5. Department of Anesthesiology and Perioperative Medicine, Queen’s University, Kingston, ON, Canada
  6. Department of Pediatrics, Division of Emergency Medicine, Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada
  7. Department of Anesthesia, The Civic Hospital, University of Ottawa, Ottawa, ON, Canada
  8. Department of Pediatrics, Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada
  9. Departments of Anesthesia, Medicine and Surgery AIME, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada
  

文摘

Purpose Our objective was to develop and evaluate a Generic Integrated Objective Structured Assessment Tool (GIOSAT) to integrate Medical Expert and intrinsic (non-medical expert) CanMEDS competencies with non-technical skills for crisis simulation. Methods An assessment tool was designed and piloted using two pediatric anesthesia scenarios (laryngospasm and hyperkalemia). Following revision of the tool, we used previously recorded videos of anesthesia residents (n?=?50) who managed one of two intraoperative advanced cardiac life support (ACLS) scenarios (ventricular tachycardia or ventricular fibrillation). Four independent trained raters, blinded to the residents-level of training, analyzed the video recordings using the GIOSAT scale. Inter-rater reliability was calculated using intraclass correlations (ICCs) for single raters (single measure) and the average of the four raters (average measure), and construct validity was investigated by correlating GIOSAT scores with postgraduate year of residency (PGY). Results Total GIOSAT scores for the ACLS scenarios had single measure ICCs of 0.62 and average measure ICCs of 0.85. Inter-rater reliability was substantial for both Medical Expert and intrinsic competencies (single measure ICCs 0.69 and 0.62, respectively; average measure ICCs 0.90 and 0.82, respectively). We found significant correlations between PGY level and total GIOSAT score (r?=?0.36; P?=?0.011) and between PGY level and Medical Expert competencies (r?=?0.42; P?=?0.003); however, correlations were not found between PGY level and intrinsic CanMEDS competencies (r?=?0.24; P?=?0.09). Conclusion Inter-rater reliability of the total GIOSAT scores using four trained raters was substantial. Significant correlation between PGY and (i) total GIOSAT score and (ii) Medical Expert competencies supports construct validity. Evidence of validity was not obtained for intrinsic CanMEDS competencies.