Data Aggregation: A Proposed Psychometric IPD Meta-Analysis
- 1 Institute of Education, University of Zurich, Zurich, Switzerland
Abstract
Individual participant data (IPD) meta-analysis was developed to overcome several meta-analytical pitfalls of classical meta-analysis. One advantage of classical psychometric meta-analysis over IPD meta-analysis is the corrections of the aggregated unit of studies, namely study differences, i.e. , artifacts, such as measurement error. Without these corrections on a study level, meta-analysts may assume moderator variables instead of artifacts between studies. The psychometric correction of the aggregation unit of individuals in IPD meta-analysis has been neglected by IPD meta-analysts thus far. In this paper, we present the adaptation of a psychometric approach for IPD meta-analysis to account for the differences in the aggregation unit of individuals to overcome differences between individuals. We introduce the reader to this approach using the aggregation of lens model studies on individual data as an example, and lay out different application possibilities for the future (e.g., big data analysis). Our suggested psychometric IPD meta-analysis supplements the meta-analysis approaches within the field and is a suitable alternative for future analysis.
- Eysenck, H.J. (1952) The Effects of Psychotherapy: An Evaluation. Journal of Consulting Psychology, 16, 319-324. https://doi.org/10.1037/h0063633
- Smith, M.L. and Glass, G.V. (1977) Meta-Analysis of Psychotherapy Outcome Studies. American Psychologist, 32, 752-760. https://doi.org/10.1037/0003-066X.32.9.752
- Lipsey, M.W. and Wilson, D.B. (1993) The Efficacy of Psychological, Educational, and Behavioral Treatment: Confirmation from Meta-Analysis. American Psychologist, 48, 1181-1209. https://doi.org/10.1037/0003-066X.48.12.1181
- Glass, G.V. (1976) Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5, 3-8. https://doi.org/10.3102/0013189X005010003
- Glass, G.V. (2016) One Hundred Years of Research: Prudent Aspirations. Educational Researcher, 45, 69-72. https://doi.org/10.3102/0013189X16639026
- Rosenthal, R. and DiMatteo, M.R. (2001) Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews. Annual Review of Psychology, 52, 59-82. https://doi.org/10.1146/annurev.psych.52.1.59
- Schmidt, F.L. and Hunter, J.E. (2014) Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Sage, Los Angeles.
- Shadish, W.R. (2015) Introduction to the Special Issue on the Origins of Modern Meta-Analysis. Research Synthesis Methods, 6, 219-220. https://doi.org/10.1002/jrsm.1148
- Rothstein, H.R. (2008) Publication Bias as a Threat to the Validity of Meta-Analytic Results. Journal of Experimental Criminology, 4, 61-81. https://doi.org/10.1007/s11292-007-9046-9
- Robinson, W.S. (1950) Ecological Correlations and the Behavior of Individuals. American Sociological Review, 15, 351-357. https://doi.org/10.1093/ije/dyn357
- Kaufmann, E., Reips, U.-D. and Maag Merki, K. (2016) Avoiding Methodological Biases in Meta-Analysis: Use of Online Versus Offline Individual Participant Data (IPD) in Educational Psychology. Zeitschrift für Psychologie, 224, 157-167. https://doi.org/10.1027/2151-2604/a000251
- Viechtbauer, W. (2007) Accounting for Heterogeneity via Random-Effects Models and Moderator Analyses in Meta-Analysis. Zeitschrift fur Psychologie, 215, 104-121. https://doi.org/10.1027/0044-3409.215.2.104
- Chalmers, I. (1993) The Cochrane Collaboration: Preparing, Maintaining, and Disseminating Systematic Reviews of the Effects of Health Care. Annals of the New York Academy of Sciences, 703, 156-163. https://doi.org/10.1111/j.1749-6632.1993.tb26345.x