Process-Oriented Understanding Estimation Using Code Puzzles

Abstract

In the current programming education, in order to assess the true ability of learners, instructors still have no choice but to monitor their answering process, standing by them. However, this is impractical for freshman training in educational institutions and newcomer training in companies. Because of the practicality, a large number of learners are assessed at once using written tests or Web tests. They usually inquire of learners whether they know algorithms and grammar. If not, they assess only the behavior of source codes they submit, at best. Under the training based on such assessment, in reality, not a few learners fail to acquire the skill of writing source codes. It implies that the attainment of programming skills cannot be assessed only by tests on knowledge and submitted source codes. This paper proposes a method for analyzing learners’ understanding that focuses on their thinking process of pr ogramming. The proposed method focuses on a code puzzle in which learners arrange fragments of a program code to satisfy given requirements. It aims to estimate the learner’s perspective on how fragments are built up to achieve the requirements. Learners with low understanding are assumed to be different from those with high in terms of the consistency of arranging ways to compose code fragments for specific blocks in source codes. For the discrimination, the method builds a model using a hidden Markov model. The internal state obtained from this model would help instructors grasp the learner’s understanding level. The results of an experiment present that the hidden Markov model produces meaningful values, which enable instructors to interpret the understanding of individual learners.