When passing fails: Designing multiple choice assessments to control for false positives

详细信息    查看全文

• 作者：David N. Dubins ; PhD ; BASc^a ; ^{d.dubins@utoronto.ca" class="auth_mail" title="E-mail the corresponding author} ; Gregory M.K. Poon ; PhD ; BScPhm^b ; ^c ; Lalitha Raman-Wilms ; BSc(Phm) ; PharmD^a
• 关键词：MCQ ; multiple choice question
• 刊名：Currents in Pharmacy Teaching and Learning
• 出版年：2016
• 出版时间：September-October 2016
• 年：2016
• 卷：8
• 期：5
• 页码：598-608
• 全文大小：320 K

文摘

To model the quantitative probabilistic features of multiple choice question (MCQ) style assessments, with specific focus on controlling the false-positive rate (α) of wrongly passing a student, and to examine and summarize MCQ-writing tips to minimize key errors in MCQ test design.

Method

We generated Monte Carlo simulations and binomial probability distributions for different MCQ test structures, varying in number of questions, choices per question, and pass mark. Educated guessing and student blunder (incorrect response despite knowing the material) were modeled. Knowledge levels associated with α<5% were determined. Pass levels were then recalculated for each test to ensure 50% and 60% levels of student knowledge, at a <5% false-positive rate.

Results

Pass marks designed to detect failing levels of knowledge at <5% error rate varied strongly with MCQ test structure. Test specificity increased with test length, number of choices per question, and pass mark, revealing that pass marks should be customized to individual test structure. A discussion on appropriate MCQ structure and content is presented.

Conclusion

Pass marks need to address MCQ test structure and the probabilistic nature of MCQ testing to accurately discriminate learner competency.