Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study
详细信息 查看全文
- 作者：Nadia M. Bajwa ; Rachel Yudkowsky ; Dominique Belli…
- 关键词：Assessment ; Professionalism ; Resident admissions ; Validity
- 刊名：Advances in Health Sciences Education
- 出版年：2017
- 出版时间：March 2017
- 年：2017
- 卷：22
- 期：1
- 页码：69-89
- 全文大小：
- 刊物类别：Humanities, Social Sciences and Law
- 刊物主题：Medical Education;
- 出版者：Springer Netherlands
- ISSN：1573-1677
- 卷排序：22
文摘
The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane’s formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.