Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT)
详细信息 查看全文
- 作者：Ian B Puddey (6)
Annette Mercer (6)
David Andrich (7)
Irene Styles (7) - 刊名：BMC Medical Education
- 出版年：2014
- 出版时间：December 2014
- 年：2014
- 卷：14
- 期：1
- 全文大小：296 KB
- 参考文献：1. Hausknecht JP, Halpert JA, Di Paolo NT, Meghan O, Moriarty G: Retesting in selection: a meta-analysis of coaching and practice effects for tests of cognitive ability. / J Appl Psychol 2007, 92:373-85. CrossRef
2. Hunt M, Pywell S, Le L, Lay D: UMAT2012. Report on the 2012 undergraduate medicine and health sciences admission test. 2012. [ / A report prepared for the Australian Council of Education Research]
3. Griffin B, Carless S, Wilson I: The effect of commercial coaching on selection test performance. / Med Teach 2013, 35:295-00. CrossRef
4. Wilkinson TM, Wilkinson TJ: Preparation courses for a medical admissions test: effectiveness contrasts with opinion. / Med Educ 2013, 47:417-24. CrossRef
5. Mercer A, Chiavaroli N: UMAT: A validity study. A review of the underlying constructs and an analysis of the content of the undergraduate medicine and health sciences admission test. 2006.
6. Edwards D, Friedman T, Coates H: Establishing the criterion validity of UMAT: A multi-cohort study of Australian medical students. 2011.
7. ABS series 1267.0 - Australian standard classification of languages (ASCL) 2011. [http://www.abs.gov.au/ausstats/abs@.nsf/mf/1267.0]
8. Measuring remoteness: accessibility/remoteness index of Australia (ARIA) revised edition. Occasional papers: new series Number 14 2001. [http://www.health.gov.au/internet/main/publishing.nsf/Content/healthhistoricpubs-hfsocc-ocpanew14a.htm]
9. Lievens F, Buyse T, Sackett PR: Retest effects in operational selection settings: development and test of a framework. / Pers Psychol 2005, 58:981-007. CrossRef
10. Lievens F, Reeve CL, Heggestad ED: An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. / J Appl Psychol 2007, 92:1672-682. CrossRef
11. Hausknecht JP, Trevor CO, Farr JL: Retaking ability tests in a selection setting: implications for practice effects, training performance, and turnover. / J Appl Psychol 2002, 87:243-54. CrossRef
12. Te Nijenhuis J, Voskuijl OF, Schijve NB: Practice and coaching on IQ tests: quite a lot of g . / Int J Sel Assess 2001, 9:302-08. CrossRef
13. Reeve CL, Lam H: The psychometric paradox of practice effects due to retesting: measurement invariance and stable ability estimates in the face of observed score changes. / Intelligence 2005, 33:535-49. CrossRef
14. McGaghie WC, Downing SM, Kubilius R: What is the impact of commercial test preparation courses on medical examination performance? / Teach Learn Med 2004, 16:202-11. CrossRef
15. Griffin B, Harding DW, Wilson IG, Yeomans ND: Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. / MJA 2008, 189:270-73.
16. Chomsky N: / Aspects of the theory of syntax. Cambridge, Massachusetts: MIT Press; 1976.
17. Casey MB, Brabeck MM: Exceptions to the male advantage on a spatial task: family handedness and college major as factors identifying women who excel. / Neuropsychologia 1989, 27:689-96. CrossRef
18. Andrich D, Styles I: Psychometric evidence of intellectual growth spurts in early adolescence. / J Early Adolesc 1994, 14:328-44. CrossRef
19. Powers DE: Relations of test item characteristics to test preparation/test practice effects: a quantitative summary. / Psychol Bull 1986, 100:67-7. CrossRef
20. Waller MI: Modeling guessing behavior: a comparison of two IRT models. / Appl Psychol Meas 1989, 13:233-43. CrossRef
21. Andrich D, Marais I, Humphry S: Using a theorem by Andersen and the dichotomous Rasch model to assess the presence of random guessing in multiple choice items. / J Educ Behav Stat 2012, 37:417-42. CrossRef
22. Puddey IB, Mercer A, Carr SE, Louden W: Potential influence of selection criteria on the demographic composition of students in an Australian medical school. / BMC Med Educ 2011, 11:97. CrossRef
23. The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6920/14/48/prepub - 作者单位：Ian B Puddey (6)
Annette Mercer (6)
David Andrich (7)
Irene Styles (7)
6. Faculty of Medicine, Dentistry and Health Sciences, University of Western Australia, 35 Stirling Hwy, Crawley, WA, 6009, Australia
7. Graduate School of Education, University of Western Australia, 35 Stirling Hwy, Crawley, WA, 6009, Australia - ISSN：1472-6920
文摘
Background The UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections -logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants of any practice effects on the UMAT and their implications for equity in subsequent selection processes. Methods Between 2000 and 2012, 158,909 UMAT assessments were completed. From these, 135,833 cases were identified where a candidate had sat once or more during that period with 117,505 cases (86.5%) having sat once, 14,739 having sat twice (10.9%), 2,752 thrice (2%) and 837, 4 or more times (0.6%). Subsequent analyses determined predictors of multiple re-sits as well as the magnitude and socio-demographic determinants of any practice effects. Results Increased likelihood of re-sitting the UMAT twice or more was predicted by being male, of younger age, being from a non-English language speaking background and being from NZ and for Australian candidates, being urban rather than rurally based. For those who sat at least twice, the total UMAT score between a first and second attempt improved by 10.7 ± 0.2 percentiles, UMAT-1 by 8.3 ± 0.2 percentiles, UMAT-2 by 8.3 ± 0.2 percentiles and UMAT-3 by 7.7 ± 0.2 percentiles. An increase in total UMAT percentile score on re-testing was predicted by a lower initial score and being a candidate from NZ rather than from Australia while a decrease was related to increased length of time since initially sitting the test, older age and non-English language background. Conclusions Re-sitting the UMAT augments performance in each of its components together with the total UMAT percentile score. Whether this increase represents just an improvement in performance or an improvement in understanding of the variables and therefore competence needs to be further defined. If only the former, then practice effects may be introducing inequity in student selection for medical or dental schools in Australia or NZ.