Measuring Creativity: A Case Study Probing Rubric Effectiveness for Evaluation of Project-Based Learning Solutions

Abstract

This research investigation focused upon whether creativity in project outcomes can be consistently measured through assessment tools, such as rubrics. Our case study research involved student-development of landscape design solutions for the Tennessee Williams Visitors Center. Junior and senior level undergraduates (N = 40) in landscape architecture design classes were assigned into equitable groups (n = 11) by an educational psychologist. Groups were subsequently assigned into either a literary narrative or abstract treatment classroom. We investigated whether student groups who were guided in their project development with abstract treatments were more likely to produce creative abstract design solutions when compared to those student groups who were guided with literary narrative interpretations. Final design solutions were presented before an audience and a panel of jurors (n = 9), who determined the outstanding project solutions through the use of a rubric, custom-designed to assess the project outcomes. Although our assumption was that the measurement of the creativity of groups’ designs would be consistent through the use of the rubric, we uncovered some discrepancies between rubric score sheets and jurors’ top choices. We subjected jurors’ score sheets and results to a thorough analysis, and four persistent themes emerged: 1) Most jurors did not fully understand the rubric’s use, including the difference between dichotomous categories and scored topics; 2) Jurors were in agreement that 6 of the 11 projects scored were outstanding submissions; 3) Jurors who had directly worked with a classroom were more likely to score that class’ groups higher; and 4) Most jurors, with the exception of two raters, scored the abstract treatment group projects as higher and more creative. We propose that while the rubric appeared to be effective in assessing creative solutions, a more thorough introduction to its use is warranted for jurors. More research is also needed as to whether prior interaction with student groups influences juror ratings.