Validity Evidence for Assessments on a UK Graduate Entry Medical Course

Abstract

Graduate entry medical courses (GEC) have been introduced in to the UK to increase the supply of doctors and to widen participation. In addition to evaluation against these outcomes, the educational process should also be evaluated. One aspect of process is assessment and different types of validity evidence for the assessments used should be provided. This paper provides validity evidence for the assessments on a UK GEC, focusing on the 2010/11 assessment diet. The types of validity evidence provided are content, internal structure, relationship with other variables and consequences. Students’ GEC assessment results are used to determine whether or not students should progress to Year 3 on the traditional course. 66% of the learning outcome/body system combinations in the assessment specification for Years 1 & 2 of the traditional course were assessed in one assessment diet. Short answer questions performed “ best ” in terms of difficulty and discrimination. The reliability of three modules was just outside the recommended range of 0.7 to 0.9. GEC performance is at least as good a predictor of final year performance as Year 1/2 performance on the traditional course. Across the six written modules for 2010/11, 12 scores (5%) were in the borderline range. Judgement regarding the validity of interpretations made from GEC assessment results is left to the reader since such judgements should not be made by those providing the validity evidence. Similar studies should aim to use benchmarks to enable results to be more objectively evaluated.