The Effect of Scrambling Test Item on Students’ Performance and Difficulty Level of MCQs Test in a College of Medicine, KKU
- 1 Department of Obstetrics & Gynecology, College of Medicine, King Khalid University, Abha, KSA
- 2 Department of Medical Education and Obstetrics & Gynecology, College of Medicine, King Khalid University, Abha, KSA
- 3 Department of Medical Education, College of Medicine, King Khalid University, Abha, KSA
- 4 Department of Medical Education, College of Medicine, King Khalid University, Abha, KSA
- 5 Department of Biochemistry and Medical Education, King Khalid University, Abha, KSA
Abstract
Background: Multiple Choice Tests (MCQs) are commonly used assessment tool in medical schools, which is delivered to our student in four versions (A, B, C and D) to avoid cheating. The aim of this study was to investigate the effect of scrambling test items on students’ performance and the difficulty level of each version, so as to decide on continuing randomization of test items or keeping it without randomization. Methods: A prospective, cross-sectional study was conducted, the participants were the 5 th year undergraduate medical students during their major course of obstetrics and gynecology. Three tests where items were randomized a re delivered to the students. After correction, the marks obtained by the candidates and difficulty index of each version were entered into the Statistical Package for Social Sciences (SPSS) version 20 and comparison amongst these four versions w as carried out through analysis of variance (ANOVA). A p -value of <0.05 was considered as statistically significant. Results : No significant difference was found in the mean difficulty index for different versions in each test and there are no statistically different results when we compared version A mean students’ scores to other versions (B, C and D) after applying ANOVA analysis (F = 1.14, p = 0.42), (F = 0.75, p = 0.69) and (F = 1.29, p = 0.34) ; (F = 0.84, p = 0.62), (F = 0.81, p = 0.64) and (F = 0.62, p = 0.79) ; (F = 0.62, p = 0.79), (F = 0.35, p = 0.95) and (F = 0.83, p = 0.64) for test 1, 2 and 3 respectively. Conclusions: Our study revealed that randomization of MCQs test item into versions to avoid cheating does not affect student performance or the difficulty level of the exam.
- Al Mahmoud, T., Elzubeir, M., Shaban, S., & Branicki, F. (2015). An Enhancement-Focused Framework for Developing High Quality Single Best Answer Multiple Choice Questions. Education for Health, 28, 194-200. https://doi.org/10.4103/1357-6283.178604
- Di Battista, D., & Kurzawa, L. (2011). Examination of the Quality of Multiple-Choice Items on Classroom Tests. Canadian Journal for the Scholarship of Teaching and Learning, 2, 4. https://doi.org/10.5206/cjsotl-rcacea.2011.2.4
- Doerner, W., & Calhoun, J. (2009). The Impact of the Order of Test Questions in Introductory Economics. https://doi.org/10.2139/ssrn.1321906
- Khan, J. S., Tabasum, S., Mukhtar, O., & Iqbal, M. (2013). The Effect on Student Performance of Scrambling Questions and Their Stems in Medical Colleges Admission Tests. Journal of the College of Physicians and Surgeons Pakistan, 23, 904-906.
- Ollennu, S. N. N. (2015). The Impact of Item Position in Multiple-Choice Test on Student Performance at the Basic Education Certificate Examination (BECE) Level: University of Cape Coast. Universal Journal of Educational Research, 3, 718-723. https://doi.org/10.13189/ujer.2015.031009
- Raux, M., Sangnier, M., & Van Ypersele, T. (2017). Scrambled Questions Penalty in Multiple Choice Tests: New Evidence from French Undergraduate Students. Economics Bulletin, 37, 347-351.
- Schroeder, J., Murphy, K. L., & Holme, T.A. (2012). Investigating Factors That Influence Item Performance on ACS Exams. Journal of Chemical Education, 89, 346-350. https://doi.org/10.1021/ed101175f
- Soureshjani, K. H. (2011). Item Sequence on Test Performance: Easy Items First? Language Testing in Asia, 1, 46-59. https://doi.org/10.1186/2229-0443-1-3-46
- Sue, D. L. (2009). The Effect of Scrambling Test Questions on Student Performance in a Small Class Setting. Journal for Economic Educators, 9, 32-41.
- Zaman, A., Niwaz, A., Faize, F., & Dahar, M. (2010). Analysis of Multiple Choice Items and the Effect of Items’ Sequencing on Difficulty Level in the Test of Mathematics. European Journal of Social Sciences, 17, 61-67.