Challenging script concordance test reference standard by evidence: do judgments by emergency medicine consultants agree with likelihood ratios?
详细信息 查看全文
- 作者：Seyed-Foad Ahmadi (1) (2)
Shahin Khoshkish (1) (3)
Kamran Soltani-Arabshahi (1)
Peyman Hafezi-Moghadam (4)
Golara Zahmatkesh (1)
Parisa Heidari (1) (5)
Davood Baba-Beigloo (6)
Hamid R Baradaran (1)
Shahram Lotfipour (7)
1. Center for Educational Research in Medical Sciences ; Iran University of Medical Sciences ; Tehran ; 14496 ; Iran
2. Program in Public Health ; Department of Population Health and Disease Prevention ; University of California Irvine ; 653 E. Peltason Dr. ; Irvine ; CA ; 92697 ; USA
3. Klinik f眉r Innere Medizin III ; Kardiologie ; Angiologie und Internistische Intensivmedizin ; Universit盲tsklinikum des Saarlandes ; Homburg/Saar ; 66421 ; Germany
4. Department of Emergency Medicine ; Iran University of Medical Sciences ; Tehran ; 14496 ; Iran
5. Department of Neurology ; Saarland University Medical Center ; Homburg/Saar ; 66421 ; Germany
6. Kamyar Clinic ; Tehran ; 51406 ; Iran
7. Department of Emergency Medicine ; School of Medicine ; University of California Irvine ; Irvine ; CA ; 92697 ; USA - 关键词：Clinical judgment ; Script concordance test ; Likelihood ratio ; Visual analog scales ; Evidence ; based medicine ; Diagnosis ; Decision ; making
- 刊名：International Journal of Emergency Medicine
- 出版年：2014
- 出版时间：December 2014
- 年：2014
- 卷：7
- 期：1
- 全文大小：469 KB
- 参考文献：1. Boulouffe, C, Doucet, B, Muschart, X, Charlin, B, Vanpee, D (2013) Assessing clinical reasoning using a script concordance test with electrocardiogram in an emergency medicine clerkship rotation. Emerg Med J 31: pp. 313-316 2012-201737" target="_blank" title="It opens in new window">CrossRef
2. Humbert, AJ, Besinger, B, Miech, EJ (2011) Assessing clinical reasoning skills in scenarios of uncertainty: convergent validity for a script concordance test in an emergency medicine clerkship and residency. Acad Emerg Med 18: pp. 627-634 2712.2011.01084.x" target="_blank" title="It opens in new window">CrossRef
3. Carriere, B, Gagnon, R, Charlin, B, Downing, S, Bordage, G (2009) Assessing clinical reasoning in pediatric emergency medicine: validity evidence for a script concordance test. Ann Emerg Med 53: pp. 647-652 2008.07.024" target="_blank" title="It opens in new window">CrossRef
4. Park, AJ, Barber, MD, Bent, AE, Dooley, YT, Dancz, C, Sutkin, G, Jelovsek, JE (2010) Assessment of intraoperative judgment during gynecologic surgery using the script concordance test. Am J Obstet Gynecol 203: pp. 240
5. Mathieu, S, Couderc, M, Glace, B, Tournadre, A, Malochet-Guinamand, S, Pereira, B, Dubost, J-J, Soubrier, M (2013) Construction and utilization of a script concordance test as an assessment tool for dcem3 (5th year) medical students in rheumatology. BMC Med Educ 13: pp. 166 2-6920-13-166" target="_blank" title="It opens in new window">CrossRef
6. Duggan, P, Charlin, B (2012) Summative assessment of 5th year medical students' clinical reasoning by script concordance test: requirements and challenges. BMC Med Educ 12: pp. 29 2-6920-12-29" target="_blank" title="It opens in new window">CrossRef
7. Nouh, T, Boutros, M, Gagnon, R, Reid, S, Leslie, K, Pace, D, Pitt, D, Walker, R, Schiller, D, MacLean, A, Hameed, M, Fata, P, Charlin, B, Meterissian, SH (2012) The script concordance test as a measure of clinical reasoning: a national validation study. Am J Surg 203: pp. 530-534 2011.11.006" target="_blank" title="It opens in new window">CrossRef
8. Piovezan, RD, Cust贸dio, O, Cendoroglo, MS, Batista, NA, Lubarsky, S, Charlin, B (2012) Assessment of undergraduate clinical reasoning in geriatric medicine: application of a script concordance test. J Am Geriatr Soc 60: pp. 1946-1950 2-5415.2012.04152.x" target="_blank" title="It opens in new window">CrossRef
9. Bursztejn, AC, Cuny, JF, Adam, JL, Sido, L, Schmutz, JL, Korwin, JD, Latarche, C, Braun, M, Barbaud, A (2011) Usefulness of the script concordance test in dermatology. J Eur Acad Dermatol Venereol 25: pp. 1471-1475 2011.04008.x" target="_blank" title="It opens in new window">CrossRef
10. Humbert, AJ, Johnson, MT, Miech, E, Friedberg, F, Grackin, JA, Seidman, PA (2011) Assessment of clinical reasoning: a script concordance test designed for pre-clinical medical students. Med Teach 33: pp. 472-477 2159X.2010.531157" target="_blank" title="It opens in new window">CrossRef
11. Kania, RE, Verillaud, B, Tran, H, Gagnon, R, Kazitani, D, Huy, PTB, Herman, P, Charlin, B (2011) Online script concordance test for clinical reasoning assessment in otorhinolaryngology: the association between performance and clinical experience. Arch Otolaryngol Head Neck Surg 137: pp. 751-755 2011.106" target="_blank" title="It opens in new window">CrossRef
12. Lambert, C, Gagnon, R, Nguyen, D, Charlin, B (2009) The script concordance test in radiation oncology: validation study of a new tool to assess clinical reasoning. Radiat Oncol 4: pp. 7 CrossRef
13. Lubarsky, S, Chalk, C, Kazitani, D, Gagnon, R, Charlin, B (2009) The script concordance test: a new tool assessing clinical judgement in neurology. Can J Neurol Sci 36: pp. 326-331
14. Meterissian, SH (2006) A novel method of assessing clinical reasoning in surgical residents. Surg Innov 13: pp. 115-119 291042" target="_blank" title="It opens in new window">CrossRef
15. Fournier, JP, Demeester, A, Charlin, B (2008) Script concordance tests: guidelines for construction. BMC Med Inform Decis Mak 8: pp. 18 2-6947-8-18" target="_blank" title="It opens in new window">CrossRef
16. Bowen, JL (2006) Educational strategies to promote clinical diagnostic reasoning. N Engl J Med 355: pp. 2217-2225 2" target="_blank" title="It opens in new window">CrossRef
17. Oxman, AD, Guyatt, GH (1993) The science of reviewing research. Ann N Y Acad Sci 703: pp. 125-133 2.1993.tb26342.x" target="_blank" title="It opens in new window">CrossRef
18. Zehtabchi, S, Kline, JA (2010) The art and science of probabilistic decision-making in emergency medicine. Acad Emerg Med 17: pp. 521-523 2712.2010.00739.x" target="_blank" title="It opens in new window">CrossRef
19. Simel, DL, Rennie, D (2009) Rational clinical examination: Evidence-based clinical diagnosis. McGraw-Hill, Chicago
20. Laerhoven, H, Zaag-Loonen, HJ, Derkx, BHF (2004) A comparison of Likert scale and visual analogue scales as response options in children's questionnaires. Acta Paediatr 93: pp. 830-835 2227.2004.tb03026.x" target="_blank" title="It opens in new window">CrossRef
21. Guyatt, GH, Townsend, M, Berman, LB, Keller, JL (1987) A comparison of Likert and visual analogue scales for measuring change in function. J Chronic Dis 40: pp. 1129-1133 21-9681(87)90080-4" target="_blank" title="It opens in new window">CrossRef
22. Straus, SERW, Glasziou, P, Haynes, RB (2005) Diagnosis and screening. Evidence-based medicine: how to practice and teach EBM. Elsevier, London, pp. 67-99
23. Hampton, JR, Harrison, MJ, Mitchell, JR, Prichard, JS, Seymour, C (1975) Relative contributions of history-taking, physical examination, and laboratory investigation to diagnosis and management of medical outpatients. Br Med J 2: pp. 486-489 2.5969.486" target="_blank" title="It opens in new window">CrossRef
24. Petrucci, AM, Nouh, T, Boutros, M, Gagnon, R, Meterissian, SH (2013) Assessing clinical judgment using the script concordance test: the importance of using specialty-specific experts to develop the scoring key. Am J Surg 205: pp. 137-140 2012.09.002" target="_blank" title="It opens in new window">CrossRef
25. Gagnon, R, Lubarsky, S, Lambert, C, Charlin, B (2011) Optimization of answer keys for script concordance testing: should we exclude deviant panelists, deviant responses, or neither?. Adv Health Sci Educ Theory Pract 16: pp. 601-608 279-2" target="_blank" title="It opens in new window">CrossRef
26. Charlin, B, Gagnon, R, Lubarsky, S, Lambert, C, Meterissian, S, Chalk, C, Goudreau, J, Vleuten, C (2010) Assessment in the context of uncertainty using the script concordance test: more meaning for scores. Teach Learn Med 22: pp. 180-186 2010.488197" target="_blank" title="It opens in new window">CrossRef
27. Charlin, B, Gagnon, R, Pelletier, J, Coletti, M, Abi-Rizk, G, Nasr, C, Sauve, E, Vleuten, C (2006) Assessment of clinical reasoning in the context of uncertainty: the effect of variability within the reference panel. Med Educ 40: pp. 848-854 2929.2006.02541.x" target="_blank" title="It opens in new window">CrossRef
28. Straus, SE, Glasziou, P, Richardson, WS, Haynes, RB (2010) Introduction. Evidence-based medicine: How to practice and teach it. Churchill Livingstone, Edinburgh, pp. 1-12
29. Ramos, K, Linscheid, R, Schafer, S (2003) Real-time information-seeking behavior of residency physicians. Fam Med 35: pp. 257-260
30. Graber, M, Gordon, R, Franklin, N (2002) Reducing diagnostic errors in medicine: what's the goal?. Acad Med 77: pp. 981-992 200210000-00009" target="_blank" title="It opens in new window">CrossRef
31. Norman, GR, Eva, KW (2010) Diagnostic error and clinical reasoning. Med Educ 44: pp. 94-100 2923.2009.03507.x" target="_blank" title="It opens in new window">CrossRef
32. Nendaz, M, Perrier, A (2012) Diagnostic errors and flaws in clinical reasoning: mechanisms and prevention in practice. Swiss Med Wkly 142: pp. w13706
33. Lineberry, M, Kreiter, CD, Bordage, G (2013) Threats to validity in the use and interpretation of script concordance test scores. Med Educ 47: pp. 1175-1183 2283" target="_blank" title="It opens in new window">CrossRef - 刊物主题：Emergency Medicine; Pediatrics; Internal Medicine; Angiology; Cardiology;
- 出版者：Springer Berlin Heidelberg
- ISSN：1865-1380
文摘
Background We aimed to compare the clinical judgments of a reference panel of emergency medicine academic physicians against evidence-based likelihood ratios (LRs) regarding the diagnostic value of selected clinical and paraclinical findings in the context of a script concordance test (SCT). Findings A SCT with six scenarios and five questions per scenario was developed. Subsequently, 15 emergency medicine attending physicians (reference panel) took the test and their judgments regarding the diagnostic value of those findings for given diseases were recorded. The LRs of the same findings for the same diseases were extracted from a series of published systematic reviews. Then, the reference panel judgments were compared to evidence-based LRs. To investigate the test-retest reliability, five participants took the test one month later, and the correlation of their first and second judgments were quantified using Spearman rank-order coefficient. In 22 out of 30 (73.3%) findings, the expert judgments were significantly different from the LRs. The differences included overestimation (30%), underestimation (30%), and judging the diagnostic value in an opposite direction (13.3%). Moreover, the score of a hypothetical test-taker was calculated to be 21.73 out of 30 if his/her answers were based on evidence-based LRs. The test showed an acceptable test-retest reliability coefficient (Spearman coefficient: 0.83). Conclusions Although SCT is an interesting test to evaluate clinical decision-making in emergency medicine, our results raise concerns regarding whether the judgments of an expert panel are sufficiently valid as the reference standard for this test.