The Similarity Index: A New and Simple Algorithm to Measure Coherence in Polls <br/>—A Case Study Where the Students Evaluate Assessments in a General Chemistry Course — Oak Academic Publishing
Research ArticleOpen AccessGoogle Scholar indexed
The Similarity Index: A New and Simple Algorithm to Measure Coherence in Polls <br/>—A Case Study Where the Students Evaluate Assessments in a General Chemistry Course
Departamento de Polímeros, Universidad de Concepción, Concepción, Chile
1 Departamento de Polímeros, Universidad de Concepción, Concepción, Chile
Assessments have been a crucial part of the process of learning and teaching. Most of the time, this process has been measured with summative assessments but what about formative ones? Historically, the teacher chooses the appropriate assessment for his/her class. However, what do the students have to say about that? In this case study, we present the results of a Likert-type poll for five assessments applied in a General Chemistry course. Students gave their answers according to three measurable intended aspects: the cognitive, the emotional and the social. To measure the coherence of those three aspects a similarity index was created. This instrument, together with the Likert-type poll, allowed us to advise which assessment should be accepted or rejected according to the student’s point of view. From a variety of assessments, the student selected the following increasing raking of assessments: Short Multiple-Choice Questions, General Questions/Answers Out Loud and Prediction-Observation-Explanation.
KeywordsStudent EvaluationEvaluation MethodsFormative AssessmentSimilarity IndexFirst-Year UndergraduateChemical Education Research
Angelo, T., & Cross, P. (1993). Classroom Assessment Techniques. A Handbook for College Teachers. Jossey-Bass.
Arriaga, M. O., Villegas, G., & Arciniega, L. (2017). Evaluación del aprendizaje significativo con la aplicación Socrative. Evaluation of Meaningful Learning with the Socrative Application. Tecnología Educativa Revista Conaic, 4, 52-57. https://doi.org/10.32671/terc.v4i2.106
Bannert, M., & Mengelkamp, C. (2008). Assessment of Metacognitive Skills by Means of Instruction to Think Aloud and Reflect When Prompted. Does the Verbalisation Method Affect Learning? Metacognition and Learning, 3, 39-58. https://doi.org/10.1007/s11409-007-9009-6
Blackburn, M. (2015). I Am Not a Superhero But I Do Have Secret Weapons! Using Technology in Higher Education Teaching to Redress the Power Balance. Journal of Pedagogic Development, 5, pages. http://hdl.handle.net/10547/346546
Bresnock, A. E., Graves, P. E., & White, N. (1989). Multiple-Choice Testing—Question and Response Position. Journal of Economic Education, 20, 239-245. https://doi.org/10.1080/00220485.1989.10844626
Bronshteyn, K., & Baladad, R. (2006). Perspectives on … Librarians as Writing Instructors: Using Paraphrasing Exercises to Teach Beginning Information Literacy Students. Journal of Academic Librarianship, 32, 533-536. https://doi.org/10.1016/j.acalib.2006.05.010
Bruschi, D., Lanzi, A., & Fovino, I. N. (2007). A Protocol for Anonymous and Accurate E-Polling. In Secure E-Government Web Services (pp. 180-198). IGI Global. https://doi.org/10.4018/978-1-59904-138-4.ch011
Bulwik, M. (2004). La evaluación de los aprendizajes y el portafolios. Educación Química, 15, 104-107. https://doi.org/10.22201/fq.18708404e.2004.2.66195
Butler, A. C. (2018). Multiple-Choice Testing in Education: Are the Best Practices for Assessment Also Good for Learning? Journal of Applied Research in Memory and Cognition, 7, 323-331. https://doi.org/10.1016/j.jarmac.2018.07.002
Carifio, J., & Perla, R. J. (2007). Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and Their Antidotes. Journal of Social Sciences, 3, 106-116. https://doi.org/10.3844/jssp.2007.106.116
Chamizo, J. A. (1995). Mapas conceptuales en la ensenanza y la evaluación de la química. Educación Química, 6, 118-124. https://doi.org/10.22201/fq.18708404e.1995.2.66719
Chamizo, J. A. (1996). Evaluación de los aprendizajes en química. Segunda parte: Registros de aprendizaje, asociación de palabras y portafolios. Educación Química, 7, 86-89. https://doi.org/10.22201/fq.18708404e.1996.2.66671
Cheung, A. K. F. (2016). Paraphrasing Exercises and Training for Chinese to English Consecutive Interpreting. FORUMFORUM. Revue Internationale d’interprétation et de Traduction/International Journal of Interpretation and Translation, 14, 1-18. https://doi.org/10.1075/forum.14.1.01che
Creme, P. (2005). Should Student Learning Journals Be Assessed? Assessment & Evaluation in Higher Education, 30, 287-296. https://doi.org/10.1080/02602930500063850
Davari, M., & Bahraman, M. (2019). I Explain, Therefore I Learn: Improving Students’ Assessment Literacy and Deep Learning by Teaching. Studies in Educational Evaluation, 61, 66-73. https://doi.org/10.1016/j.stueduc.2019.03.002
Dodd, D. K., & Leal, L. (1988). Answer Justification—Removing the Trick from Multiple-Choice Questions. Teaching of Psychology, 15, 37-38. https://doi.org/10.1207/s15328023top1501_8
Fischman, G. E., Topper, A. M., Silova, I., Goebel, J., & Holloway, J. L. (2019). Examining the Influence of International Large-Scale Assessments on National Education Policies. Journal of Education Policy, 34, 470-499. https://doi.org/10.1080/02680939.2018.1460493
Frías, M. A., Arce, C., & Flores-Morales, P. (2016). Uso de la plataforma socrative.com para alumnos de Química General. Educación Química, 27, 59-66. https://doi.org/10.1016/j.eq.2015.09.003
Guarascio, A. J., Nemecek, B. D., & Zimmerman, D. E. (2017). Evaluation of Students’ Perceptions of the Socrative Application versus a Traditional Student Response System and Its Impact on Classroom Engagement. Currents in Pharmacy Teaching and Learning, 9, 808-812. https://doi.org/10.1016/j.cptl.2017.05.011
Güven, E. (2014). The Effect of Project Based Learning Method Supported by Prediction-Observation-Explanations on the Attitude and Behaviors towards Environmental Problems. Education and Science, 39, 25-39.
Hartman, T., & Schachter, E. (2019). High School Students “Talk Back” to the Anonymous Researcher behind the Questionnaire. The Journal of Experimental Education, Volume number, 1-13. https://doi.org/10.1080/00220973.2019.1573795
Harwood, W. S. (1996). The One-Minute Paper: A Communication Tool for Large Lecture Classes. Journal of Chemical Education, 73, 229. https://doi.org/10.1021/ed073p229
Havnes, A., Smith, K., Dysthe, O., & Ludvigsen, K. (2012). Formative Assessment and Feedback: Making Learning Visible. Studies in Educational Evaluation, 38, 21-27. https://doi.org/10.1016/j.stueduc.2012.04.001
Hong, J. C., Hwang, M. Y., Liu, M. C., Ho, H. Y., & Chen, Y. L. (2014). Using a “Prediction-Observation-Explanation” Inquiry Model to Enhance Student Interest and Intention to Continue Science Learning Predicted by Their Internet Cognitive Failure. Computers and Education, 72, 110-120. https://doi.org/10.1016/j.compedu.2013.10.004
Hortigüela, D., Palacios, A., & López, V. (2019). The Impact of Formative and Shared or Co-Assessment on the Acquisition of Transversal Competences in Higher Education. Assessment and Evaluation in Higher Education, 44, 933-945. https://doi.org/10.1080/02602938.2018.1530341
Hsu, C. Y., Tsai, C. C., & Liang, J. C. (2011). Facilitating Preschoolers’ Scientific Knowledge Construction via Computer Games Regarding Light and Shadow: The Effect of the Prediction-Observation-Explanation (POE) Strategy. Journal of Science Education and Technology, 20, 482-493. https://doi.org/10.1007/s10956-011-9298-z
Kearney, M. (2004). Classroom Use of Multimedia-Supported Predict-Observe-Explain Tasks in a Social Constructivist Learning Environment. Research in Science Education, 34, 427-453. https://doi.org/10.1007/s11165-004-8795-y
Kokina, J., & Juras, P. E. (2017). Using Socrative to Enhance Instruction in an Accounting Classroom. Journal of Emerging Technologies in Accounting, 14, 85-97. https://doi.org/10.2308/jeta-51700
Kose, S., & Bilen, K. (2012). The Effect of Laboratory Activities Based on a POE Strategy for Class Teacher Candidates’ Achievements, Science Process Skills and Understanding the Nature of Science. Energy Education Science and Technology Part B Social and Educational Studies, 4, 2357-2368.
Kumar, A., Kumar, M. S., & Gulia, R. (2017). The Use of Classroom Assessment Techniques (CATs) to Enhance Learning and Motivation in First Year Undergraduate Students. Journal of Research in Medical Education & Ethics, 7, 93. https://doi.org/10.5958/2231-6728.2017.00016.6
Lancaster, E. L. (2007). Questions and Answers (Encouraging Students to Sing and Count out Loud). Clavier, 46, 48.
Latifah, S., Irwandani, I., Saregar, I., Diani, R., Fiani, O., Widayanti, W., & Deta, U. A. (2019). How the Predict-Observe-Explain (POE) Learning Strategy Remediates Students’ Misconception on Temperature and Heat Materials? Journal of Physics: Conference Series, 1171, 1-6. https://doi.org/10.1088/1742-6596/1171/1/012051
Lewis, C. E. (2019). Is the Flipped Classroom a Panacea for Medical Education? Current Surgery Reports, 7, 9. https://doi.org/10.1007/s40137-019-0230-4
Likert, R. (1932). A Technique for the Measurement of Attitude. Archives of Psychology, 140, 5-55.
Lutterodt, M. C. (2017). Qualitative Formative Feedback to the Teacher. The Use of “Learn Evaluation” a Modified “One Minute Paper”. In Improving University Science Teaching and Learning—Pedagogical Projects 2017 (pp. 133-147). Department of Science Education, University of Copenhagen.
Manning, R. D., Keiper, M. C., & Jenny, S. E. (2017). Pedagogical Innovations for the Millennial Sport Management Student: Socrative and Twitter. Sport Management Education Journal, 11, 45-54. https://doi.org/10.1123/smej.2016-0014
Matas, A. (2018). Diseno del formato de escalas tipo Likert: Un estado de la cuestión. Revista Electronica de Investigacion Educativa, 20, 38-47. https://doi.org/10.24320/redie.2018.20.1.1347
Maxwell, S. R. J. (2012). An Agenda for UK Clinical Pharmacology. How Should Teaching of Undergraduates in Clinical Pharmacology and Therapeutics Be Delivered and Assessed? British Journal of Clinical, 73, 893-899. https://doi.org/10.1111/j.1365-2125.2012.04232.x
Mayowski, C. A., Norman, M. K., & Kapoor, W. N. (2018). Assessing an Assessment: The Review and Redesign of a Competency-Based Mid-Degree Evaluation. Journal of Clinical and Translational Science, 2, 223-227. https://doi.org/10.1017/cts.2018.321
Metcalfe, J., & Xu, J. (2018). Learning from One’s Own Errors and Those of Others. Bulletin & Review, 25, 402-408. https://doi.org/10.3758/s13423-017-1287-7
Moran, A. S., Swanson, H. L., Gerber, M. M., & Fung, W. (2014). The Effects of Paraphrasing Interventions on Problem-Solving Accuracy for Children at Risk for Math Disabilities. Learning Disabilities Research & Practice, 29, 97-105. https://doi.org/10.1111/ldrp.12035
Nunez-Oviedo, M. C., & Clement, J. J. (2019). Large Scale Scientific Modeling Practices That Can Organize Science Instruction at the Unit and Lesson Levels. Frontiers in Education, 4, 1-22. https://doi.org/10.3389/feduc.2019.00068
Orozco, H. O., Sosa, M. R., & Martínez, F. (2018). Didactic Models at the High Education: A Reality That Can Change. Profesorado—Revista de Curriculum y Formación Del Profesorado, 22, 405-427.
Ponce, H. R., López, M. J., & Mayer, R. E. (2012). Instructional Effectiveness of a Computer-Supported Program for Teaching Reading Comprehension Strategies. Computers and Education, 59, 1170-1183. https://doi.org/10.1016/j.compedu.2012.05.013
Prashanti, E., & Ramnarayan, K. (2019). Ten Maxims of Formative Assessment. Advances in Physiology Education, 43, 99-102. https://doi.org/10.1152/advan.00173.2018
Price, N., Stephens, A. L., Clement, J., & Nunez-Oviedo, M. C. (2017). Using Imagery Support Strategies to Develop Powerful Imagistic Models. Science Scope, 41, 40-49. https://doi.org/10.2505/4/ss17_041_04_40
Quesada, V., Gómez, M. á., Gallego, M. B., & Cubero-Ibánez, J. (2019). Should I Use Co-Assessment in Higher Education? Pros and Cons from Teachers and Students’ Perspectives. Assessment & Evaluation in Higher Education, 44, 987-1002. https://doi.org/10.1080/02602938.2018.1531970
Robertson, J. (2012). Likert-Type Scales, Statistical Methods, and Effect Sizes. Communications of the ACM, 55, 6. https://doi.org/10.1145/2160718.2160721
Rubenson, K. (2019). Assessing the Status of Lifelong Learning: Issues with Composite Indexes and Surveys on Participation. International Review of Education, 65, 295-317. https://doi.org/10.1007/s11159-019-09768-3
Ruiz-Primo, M. A. (2011). Informal Formative Assessment: The Role of Instructional Dialogues in Assessing Students’ Learning. Studies in Educational Evaluation, 37, 15-24. https://doi.org/10.1016/j.stueduc.2011.04.003
Schneider, M. C., McDonel, J. S., & DePascale, C. A. (2019). Performance Assessment and Rubric Design. In The Oxford Handbook of Assessment Policy and Practice in Music Education (Volume 1, pages). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190248093.013.27
Smith, S. (2007). How Does Student Performance on Formative Assessments Relate to Learning Assessed by Exams? Journal of College Science Teaching, 36, 28-34.
Stead, D. R. (2005). A Review of the One-Minute Paper. Active Learning in Higher Education, 6, 118-131. https://doi.org/10.1177/1469787405054237
Stewart, S., & Richardson, B. (2000). Reflection and Its Place in the Curriculum on an Undergraduate Course: Should It Be Assessed? Assessment & Evaluation in Higher Education, 25, 369-380. https://doi.org/10.1080/713611443
Such, J. M., Criado, N., & García-Fornes, A. (2015). An Active Learning Technique Enhanced with Electronic Polls. International Journal of Engineering Education, 31, 1048-1057.
Tan, C. (2019). PISA and Education Reform in Shanghai. Critical Studies in Education, 60, 391-406. https://doi.org/10.1080/17508487.2017.1285336
Tan, K. E. (2017). Using Online Discussion Forums to Support Learning of Paraphrasing. British Journal of Educational Technology, 48, 1239-1249. https://doi.org/10.1111/bjet.12491
Uemlianin, I. A. (2000). Engaging Text: Assessing Paraphrase and Understanding. Studies in Higher Education, 25, 346-358. https://doi.org/10.1080/713696160
Viera, L., Ramírez, S., Wainmier, C., & Salinas, J. (2007). Criterios y actividades para la evaluación del aprendizaje en cursos universitarios de química. Educación Química, 18, 294-302. https://doi.org/10.22201/fq.18708404e.2007.4.65876
Wardrop, J. (2012). Speaking out Loud: Muslim Women, Indian Delights and Culinary Practices in eThekwini/Durban. Social Dynamics—A Journal of African Studies, 38, 221-236. https://doi.org/10.1080/02533952.2012.717209
White, R., & Gunstone, R. (1992). Probing Understanding. The Falmer Press.