Unfounded causal claims from the internet, the fact that randomized control trials (RCTs) cannot address many critical issues, and reports that scientific studies fail replication attempts suggest reconsidering how students learn to evaluate causal claims. Traditionally, students learn RCTs are at the top of the research methods hierarchy, and that they cannot infer causation from associations (e.g., correlations). Both traditions are debatable. Students need to learn how to evaluate causal claims they encounter in daily life as well as claims supported by scientific evidence. Students will become better critical thinkers from learning a definition of evidence that applies inside and outside the psychology laboratory and how to use anecdotes, associations, and RCTs in defense of causal claims. They must learn to question all evidence and seek patterns of supporting evidence for causal claims.
KeywordsCausal ClaimsEvidenceCritical ThinkingResearch MethodsStatisticsPedagogyCorrelation and Causation
Abelson, R. P. (1995). Statistics as Principled Argument. Lawrence Erlbaum Associates.
American Psychological Association (2013). APA Guidelines for the Undergraduate Psychology Major: Version 2.0. Author. https://tinyurl.com/2c5ybmcs
Aron, A., Coups, E. J., & Aron, E. N. (2011). Statistics for the Behavioral and Social Sciences: A Brief Course (5th ed.). Pearson.
Barrett, T. (2015, February 27). Inhofe Brings Snowball on Senate Floor as Evidence Globe Is Not Warming. CNN. https://www.cnn.com/2015/02/26/politics/james-inhofe-snowball-climate-change/index.html
Biddlestone, M., Flavio Azevedo, F., & van der Linden, S. (2022). Climate of Conspiracy: A Meta-Analysis of the Consequences of Belief in Conspiracy Theories about Climate Change. Current Opinion in Psychology, 46, Article ID: 101390. https://doi.org/10.1016/j.copsyc.2022.101390
Blanco, F. (2017). Positive and Negative Implications of the Causal Illusion. Consciousness and Cognition, 50, 56-68. https://doi.org/10.1016/j.concog.2016.08.012
Boudry, M., Vlerick, M., & McKay, R. (2015). Can Evolution Get Us off the Hook? Evaluating the Ecological Defense of Human Rationality. Consciousness and Cognition, 33, 524-535. https://doi.org/10.1016/j.concog.2014.08.025
Campbell, D. T., & Stanley, J. C. (1966). Experimental and Quasi-Experimental Designs for Research. Rand McNally.
Čavojová, V., Šrol, J., & Jurkovič, M. (2020). Why Should We Try to Think like Scientists? Scientific Reasoning and Susceptibility to Epistemically Suspect Beliefs and Cognitive Biases. Applied Cognitive Psychology, 34, 85-95. https://doi.org/10.1002/acp.3595
Chance, P. (1999). Thorndike’s Puzzle Boxes and the Origins of the Experimental Analysis of Behavior. Journal of the Experimental Analysis of Behavior, 72, 433-440. https://doi.org/10.1901/jeab.1999.72-433
Darwin, C. (1964). On the Origin of Species by Charles Darwin: A Facsimile of the First Edition with an Introduction by Ernst Mayr. Harvard University Press.
Drake, R. E., Latimer, E. A., Leff, H. S., McHugo, G. J., & Burns, B. J. (2004). What Is Evidence? Child and Adolescent Psychiatric Clinics of North America, 13, 717-728. https://doi.org/10.1016/j.chc.2004.05.005
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021, December 7). Investigating the Replicability of Preclinical Cancer Biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601.sa2
Fiedler, K., & Schwarz, N. (2015). Questionable Research Practices Revisited. Social Psychological and Personality Science, 7, 45-52. https://doi.org/10.1177/1948550615612150
File Drawer Problem (2007). In N. J. Salkind (Ed.), Encyclopedia of Measurement and Statistics (Vol. 1, pp. 353-354). SAGE.
Flora, D. B. (2020). Thinking about Effect Sizes: From the Replication Crisis to a Cumulative Psychological Science. Canadian Psychology, 61, 318-330. https://doi.org/10.1037/cap0000218
Goldberg, M. H. (2019). How Often Does Random Assignment Fail? Estimates and Recommendations. Journal of Environmental Psychology, 66, Article ID: 101351. https://doi.org/10.1016/j.jenvp.2019.101351
Hatfield, J., Faunce, G. J., & Soames Job, R. F. (2006). Avoiding Confusion Surrounding the Phrase “Correlation Does Not Imply Causation”. Teaching of Psychology, 33, 49-51.
Intergovernmental Panel on Climate Change (2022). Summary for Policymakers. https://tinyurl.com/fneejxkm
Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLOS Medicine, 2, e124. https://doi.org/10.1371/journal.pmed.0020124
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling. Psychological Science, 23, 524-532. https://doi.org/10.1177/0956797611430953
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Kenny, D. A. (2019). Enhancing Validity in Psychological Research. American Psychologist, 74, 1018-1028. https://doi.org/10.1037/amp0000531
Lewandowsky, S. (2021). Climate Change Disinformation and How to Combat It. Annual Review of Public Health, 42, 1-21. https://doi.org/10.1146/annurev-publhealth-090419-102409
Lilienfeld, S. O., Ammirati, R., & Landfield, K. (2009). Giving Debiasing Away. Perspectives on Psychological Science, 4, 390-398. https://doi.org/10.1111/j.1745-6924.2009.01144.x
Lindsay, D. S. (2015). Replication in Psychological Science. Psychological Science, 26, 1827-1832. https://doi.org/10.1177/0956797615616374
Loftus, E. M. (2019). Eyewitness Testimony. Applied Cognitive Psychology, 33, 498-503. https://doi.org/10.1002/acp.3542
Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., Donohue, M. R., Foran, W., Miller, R. L., Hendrickson, T. J., Malone, S. M., Kandala, S., Feczko, E., Miranda-Dominguez, O., Graham, A. M., Earl, E. A., Perrone, A J., Cordova, M., Doyle, O., & Dosenbach, N. U. F. (2022). Reproducible Brain-Wide Association Studies Require Thousands of Individuals. Nature, 603, 654-660. https://doi.org/10.1038/s41586-022-04492-9
Martín, M., & Valiña, M. D. (2023). Heuristics, Biases and the Psychology of Reasoning: State of the Art. Psychology, 14, 264-294. https://doi.org/10.4236/psych.2023.142016
Marx, W., Haunschild, R., Thor, A., & Bornmann, L. (2017). Which Early Works Are Cited Most Frequently in Climate Change Research Literature? A Bibliometric Approach Based on Reference Publication Year Spectroscopy. Scientometrics, 110, 335-353. https://doi.org/10.1007/s11192-016-2177-x
Matute, H., Blanco, F., Yarritu, I., Díaz-Lago, M., Vadillo, M. A., & Barberia, I. (2015). Illusions of Causality: How They Bias Our Everyday Thinking and How They Could Be Reduced. Frontiers in Psychology, 6, Article No. 888. https://doi.org/10.3389/fpsyg.2015.00888
Matute, H., Yarritu, I., & Vadillo, M. A. (2011). Illusions of Causality at the Heart of Pseudoscience. British Journal of Psychology, 102, 392-405. https://doi.org/10.1348/000712610X532210
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is Psychology Suffering from a Replication Crisis? What Does “Failure to Replicate” Really Mean? American Psychologist, 70, 487-498. https://doi.org/10.1037/a0039400
Mealey, L. (2000). Sex Differences: Developmental and Evolutionary Strategies. Academic Press.
Meltzoff, J., & Cooper, H. (2018). Critical Thinking about Research: Psychology and Related Fields (2nd ed.). American Psychological Association. https://doi.org/10.1037/0000052-000
Moravec, P. L., Kim, A., & Dennis, A. L. (2018). Appealing to Sense and Sensibility: System 1 and System 2 Interventions for Fake News on Social Media. Information Systems Research, 31, 987-1006. https://doi.org/10.2139/ssrn.3269902
Morling, B. (2018). Research Methods in Psychology (3rd ed.). Norton.
Morling, B. (2021). Research Methods in Psychology (4th ed.). Norton.
National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and Replicability in Science. The National Academies Press.
Open Science Collaboration (2015). Estimating the Reproducibility of Psychological Science. Science, 349, aac4716. https://doi.org/10.1126/science.aac4716
Pearl, J. (2018). Challenging the Hegemony of Randomized Controlled Trials: A Commentary on Deaton and Cartwright. Social Science & Medicine, 210, 60-62. https://doi.org/10.1016/j.socscimed.2018.04.024
Reiss, J. (2009). Causation in the Social Sciences: Evidence, Inference, and Purpose. Philosophy of the Social Sciences, 39, 20-40. https://doi.org/10.1177/0048393108328150
Richmond, A. S, Boysen, G. A., Hudson, D. L., Gurung, R. A. R., Naufel, K. Z., Neufeld, G., Landrum, R. E., Dunn, D. S., & Beers, M. (2021). The Introductory Psychology Census: A National Study. Scholarship of Teaching and Learning in Psychology, 7, 163-180. https://doi.org/10.1037/stl0000277
Sawilowsky, S. S. (2004). Teaching Random Assignment: Do You Believe It Works? Journal of Modern Applied Statistical Methods, 3, 221-226. http://digitalcommons.wayne.edu/coe_tbf/16 https://doi.org/10.22237/jmasm/1083370980
Schmidt, F. L., & Oh, I.-S. (2016). The Crisis of Confidence in Research Findings in Psychology: Is Lack of Replication the Real Problem? Or Is It Something Else? Archives of Scientific Psychology, 4, 32-37. https://doi.org/10.1037/arc0000029
Schoolov, K. (2021, October 1). Why It’s Not Possible for the Covid Vaccines to Contain a Magnetic Tracking Chip That Connects to 5G. CNBC. https://www.cnbc.com/2021/10/01/why-the-covid-vaccines-dont-contain-a-magnetic-5g-tracking-chip.html
Schwarcz, J. (2021). Can Vaccines Make Our Body Magnetic? McGill Office for Science and Society. https://tinyurl.com/2evdeavy
Sharpe, D., & Poets, S. (2020). Meta-Analysis as a Response to the Replication Crisis. Canadian Psychology, 61, 377-387. https://doi.org/10.1037/cap0000215
Stapleton, P. (2019). Avoiding Cognitive Biases: Promoting Good Decision Making in Research Methods Courses. Teaching in Higher Education, 24, 578-586. https://doi.org/10.1080/13562517.2018.1557137
Taccone, F. S., Hites, M., & Dauby, M. (2022). From Hydroxychloroquine to Ivermectin: How Unproven “Cures” Can Go Viral. Clinical Microbiology and Infection, 28, 472-474. https://doi.org/10.1016/j.cmi.2022.01.008
Thorndike, E. L. (1911). Animal Intelligence. Hafner.
Tryon, W. W. (2016). Replication Is about Effect Size: Comment on Maxwell, Lau, and Howard (2015). American Psychologist, 71, 236-237. https://doi.org/10.1037/a0040191
Wagner, P. A. (2022). Tools for Teaching and Role-Modeling Critical Thinking. Psychology, 13, 1335-1341. https://doi.org/10.4236/psych.2022.138086
Wielkiewicz, R. M. (2016). Sustainability and Psychology (2nd ed.). Main Event Press. https://www.amazon.com/dp/B012LJ9ACQ/
Wielkiewicz, R. M. (2022). A Quick Review of Statistical Thinking (4th ed.). Main Event Press. https://www.amazon.com/Quick-Review-Statistical-Thinking-ebook/dp/B09YBYXD4L/
Wilkinson, L., & the Task Force on Statistical Inference (TFSI) (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, 54, 594-604. https://doi.org/10.1037/0003-066X.54.8.594
Worrall, J. (2010). Evidence: Philosophy of Science Meets Medicine. Journal of Evaluation in Clinical Practice, 16, 356-362. https://doi.org/10.1111/j.1365-2753.2010.01400.x