Multilevel CFA models (MLV CFA) modeling permits more sophisticated construct validity research by examining relationships among factor structures, factor loadings, and errors at different hierarchical levels. In the MLV CFA models, the latent variable or variables have two kinds of elements: 1) the between-group elements (Level 2 or higher level) and 2) the within-group elements (Level 1 of lower level). The between-group elements represent the general part of the model and the within-group element the individual part. The within-level variation includes an individual-level measurement error variance, which generally expands the impact of the within-level variation to the intraclass correlations. Multilevel CFA therefore generates results corresponding to those generated by perfectly reliable measures. If the same measurement model is specified across levels, by defining each item loading to be invariant with its across-level counterpart, the researcher can equate the factor scales across levels. Thus, the factor variances at different levels are directly comparable. The fit of this constrained MLV CFA model can be evaluated by comparing it with an unconstrained model specified with freely estimated factor loadings at each level. In the present work the steps of the above procedure are fully described and additional issues relevant to the use of MLV CFA are discussed in detail.
Ainsworth, M. D. S., Blehar, M. C., & Waters, E. (1978). Patterns of Attachment: A Psychological Study of the Strange Situation. Hillsdale, NJ: Earlbaum.
Akaike, H. (1987). Factor Analysis and AIC. Psychometrika, 52, 317-332. https://doi.org/10.1007/BF02294359
Bauer, D. J. (2003). Estimating Multilevel Linear Models as Structural Models. Journal of Educational and Behavioral Statistics, 28, 135-167. https://doi.org/10.3102/10769986028002135
Bentler, P. M. (1990). Comparative Fit Indexes in Structural Models. Psychological Bulletin, 107, 238-246. https://doi.org/10.1037/0033-2909.107.2.238
Bentler, P. M. (2005). EQS 6 Structural Equations Program Manual. Encino, CA: Multivariate Software.
Bickel, R. (2007). Multilevel Analysis for Applied Research: It’s Just Regression! New York: Guilford.
Boomsma, A. (1987). The Robustness of Maximum Likelihood Estimation in Structural Equation Modeling. In P. Cuttance, & R. Ecob (Eds.), Structural Equation Modelling by Example (pp. 160-188). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511601118.010
Bovaird, J. A. (2007). Multilevel Structural Equation Models for Contextual Factors. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling Contextual Effects in Longitudinal Studies (pp. 149-182). Mahwah, NJ: Erlbaum.
Bowlby, J. (1969). Attachment and Loss: Attachment (Vol. 1). London: Penguin.
Bowlby, J. (1973). Attachment and Loss Vol. 2. Separation: Anxiety and Anger. New York: Basic Books.
Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research (2nd ed.). New York: Guilford Publications.
Bryk, A. S., & Raudenbush, S. W. (1987). Application of Hierarchical Linear Models to Assessing Change. Psychological Bulletin, 101, 147-158. https://doi.org/10.1037/0033-2909.101.1.147
Byrne, B. M. (2012). Structural Equation Modeling with Mplus: Basic Concepts, Applications, and Programming. London: Routledge. https://doi.org/10.4324/9780203807644
Chen, Q., Kwok, O.-M., Luo, W., & Willson, V. L. (2010). The Impact of Ignoring a Level of Nesting Structure in Multilevel Growth Mixture Models: A Monte Carlo Study. Structural Equation Modeling, 17, 570-589. https://doi.org/10.1080/10705511.2010.510046
Cheung, M. W.-L., & Au, K. (2005). Applications of Multilevel Structural Equation Modeling to Cross-Cultural Research. Structural Equation Modeling, 12, 598-619. https://doi.org/10.1207/s15328007sem1204_5
Chou, C.-P., Bentler, P. M., & Pentz, M. A. (1998). Comparisons of Two Statistical Approaches to Study Growth Curves: The Multilevel Model and the Latent Curve Analysis. Structural Equation Modeling, 5, 247-266. https://doi.org/10.1080/10705519809540104
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Mahwah, NJ: Erlbaum.
Curran, P. J. (2003). Have Multilevel Models Been Structural Equation Models All Along? Multivariate Behavioral Research, 38, 529-569. https://doi.org/10.1207/s15327906mbr3804_5
Darlington, R. B., & Hayes, A. F. (2017). Regression Analysis and Linear Models: Concepts, Applications, and Implementation. New York: The Guilford Press.
de Leeuw, E. D., Hox, J. J., & Dillman, D. A. (2008). The International Handbook of Survey Methodology. New York: Taylor & Francis.
Dedrick, R. F., & Greenbaum, P. E. (2010). Multilevel Confirmatory Factor Analysis of a Scale Measuring Interagency Collaboration of Children’s Mental Health Agencies. Journal of Emotional and Behavioural Disorders, 10, 1-14. https://doi.org/10.1177/1063426610365879
Dedrick, R. F., & Greenbaum, P. E. (2011). Multilevel Confirmatory Factor Analysis of a Scale Measuring Interagency Collaboration of Children’s Mental Health Agencies. Journal of Emotional and Behavioral Disorders, 19, 27-40. https://doi.org/10.1177/1063426610365879
Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying Multilevel Confirmatory Factor Analysis Techniques to the Study of Leadership. Leadership Quarterly, 16, 149-167. https://doi.org/10.1016/j.leaqua.2004.09.009
Ecob, R., & Der, G. (2003). An Iterative Method for the Detection of Outliers in Longitudinal Growth Data Using Multilevel Models. In S. P. Reise, & N. Duan (Eds.), Multilevel Modeling: Methodological Advances, Issues, and Applications (pp. 229-254). Mahwah, NJ: Erlbaum.
Eliason, S. R. (1993). Maximum Likelihood Estimation. Newbury Park, CA: Sage. https://doi.org/10.4135/9781412984928
Epstein, N. B., Bishop, D. S., & Levin, S. (1978). The McMaster Model of Family Functioning. Journal of Marital and Family Therapy, 4, 19-31. https://doi.org/10.1111/j.1752-0606.1978.tb00537.x
Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics (4th ed.). London: Sage.
Finch, W. H., & Bolin, J. E. (2017). Multilevel Modeling Using Mplus. Boca Raton, FL: Taylor & Francis Group. https://doi.org/10.1201/9781315165882
Geiser, C. (2013). Data Analysis with Mplus (English Edition). New York: The Guilford Press.
Goldstein, H. (1987). Multilevel Models in Educational and Social Research. London: Oxford University Press.
Goldstein, H., & Browne, W. (2005). Multilevel Factor Analysis Models for Continuous and Discrete Data. In A. Maydeu-Olivares, & J. J. McArdle (Eds.), Contemporary Psychometrics: A Festschrift to Roderick P. McDonald (pp. 453-475). Mahwah, NJ: Lawrence Erlbaum Associates.
Harnqvist, K., Gustafsson, J., Muthén, B., & Nelson, G. (1994). Hierarchical Models of Ability at Class and Individual Levels. Intelligence, 18, 165-118. https://doi.org/10.1016/0160-2896(94)90026-4
Heck, R. H. (2001). Multilevel Modeling with SEM. In G. A. Marcoulides, & R. E. Schumacker (Eds.), New Developments and Techniques in Structural Equation Modeling (pp. 89-127). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Heck, R. H., & Thomas, S. L. (2009). An Introduction to Multilevel Modeling Techniques (2nd ed.). New York: Routledge.
Heck, R. H., & Thomas, S. L. (2015). An Introduction to Multilevel Modeling Techniques: MLM and SEM Approaches Using Mplus (3rd ed.). New York: Routledge. https://doi.org/10.4324/9781315746494
Hoffman, L. (2007). Multilevel Models for Examining Individual Differences in Within-Person Variation and Covariation over Time. Multivariate Behavioral Research, 42, 609-629. https://doi.org/10.1080/00273170701710072
Hox, J. J. (2002). Multilevel Analysis: Techniques and Applications. Mahwah, NJ: Lawrence Erlbaum Associates Publishers. https://doi.org/10.4324/9781410604118
Hox, J. J. (2010). Multilevel Analysis: Techniques and Applications (2nd ed.). New York: Routledge. https://doi.org/10.4324/9780203852279
Hox, J. J. (2013). Multilevel Regression and Multilevel Structural Equation Modeling. In T. D. Little (Ed.), The Oxford Handbook of Quantitative Methods (pp. 281-294). New York: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199934898.013.0014
Hox, J. J., & Maas, C. J. M. (2001). The Accuracy of Multilevel Structural Equation Modeling with Psuedobalanced Groups and Small Samples. Structural Equation Modeling, 8, 157-174. https://doi.org/10.1207/S15328007SEM0802_1
Hox, J. J., Maas, C. J. M., & Brinkhuis, M. J. S. (2010). The Effect of Estimation Method and Sample Size in Multilevel Structural Equation Modeling. Statistica Neerlandica, 64, 157-170. https://doi.org/10.1111/j.1467-9574.2009.00445.x
Hu, L.-T., & Bentler, P. M. (1999). Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Structural Equation Modeling, 6, 1-55. https://doi.org/10.1080/10705519909540118
James, D. E., Schumm, W. R., Kennedy, C. E., Grigsby, C. C., Shectman, K. L., & Nichols, C. W. (1985). Characteristics of the Kansas Parental Satisfaction Scale among Two Samples of Married Parents. Psychological Reports, 57, 163-169. https://doi.org/10.2466/pr0.1985.57.1.163
Jo, B., & Muthén, B. O. (2003). Longitudinal Studies with Intervention and Noncompliance: Estimation of Causal Effects in Growth Mixture Modeling. In S. P. Reise, & N. Duan (Eds.), Multilevel Modeling: Methodological Advances, Issues, and Applications (pp. 112-139). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Joreskog, K. G., & Sorbom, D. (1989). LISREL 7: User’s Reference Guide. Chicago, IL: Scientific Software.
Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language. Chicago, IL: Scientific Software International.
Julian, M. W. (2001). The Consequences of Ignoring Multilevel Data Structures in Nonhierarchical Covariance Modeling. Structural Equation Modeling, 8, 325-352. https://doi.org/10.1207/S15328007SEM0803_1
Kalaian, S. A., & Kasim, R. M. (2007). Hierarchical Linear Modeling. In N. J. Salkind (Ed.), Encyclopedia of Measurement and Statistics (pp. 432-435). Thousand Oaks, CA: Sage.
Kaplan, D., & Kreisman, M. B. (2000). On the Validation of Indicators of Mathematics Education Using TIMSS: An Application of Multilevel Covariance Structure Modeling. International Journal of Educational Policy, Research, and Practice, 1, 217-242.
Kaplan, D., Kim, J.-S., & Kim, S.-Y. (2009). Multilevel Latent Variable Modeling: Current Research and Recent Developments. In R. E. Millsap, & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 592-613). Thousand Oaks, CA: Sage. https://doi.org/10.4135/9780857020994.n24
Kelloway, E. K. (2015). Using Mplus for Structural Equation Modeling. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781849209366
Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling (4th ed.).
Kreft, I., & de Leeuw, J. (1998). Introducing Multilevel Modeling. Newbury Park, CA: Sage.
Kwok, O.-M., West, S. G., & Green, S. B. (2007). The Impact of Misspecifying the Within-Subject Covariance Structure in Multiwave Longitudinal Multilevel Models: A Monte Carlo Study. Multivariate Behavioral Research, 42, 557-592. https://doi.org/10.1080/00273170701540537
Kyriazos, T. A. (2018). Applied Psychometrics: The 3-Faced Construct Validation Method, a Routine for Evaluating a Factor Structure. Psychology, 9, 2044-2072. https://doi.org/10.4236/psych.2018.98117
Kyriazos, T. A., Stalikas, A., Prassa, K., & Yotsidi, V. (2018a). Can the Depression Anxiety Stress Scales Short Be Shorter? Factor Structure and Measurement Invariance of DASS-21 and DASS-9 in a Greek, Non-Clinical Sample. Psychology, 9, 1095-1127. https://doi.org/10.4236/psych.2018.95069
Kyriazos, T. A., Stalikas, A., Prassa, K., & Yotsidi, V. (2018b). A 3-Faced Construct Validation and a Bifactor Subjective Well-Being Model Using the Scale of Positive and Negative Experience, Greek Version. Psychology, 9, 1143-1175. https://doi.org/10.4236/psych.2018.95071
Kyriazos, T. A., Stalikas, A., Prassa, K., Galanakis, M., Yotsidi, V., & Lakioti, A. (2018). Psychometric Evidence of the Brief Resilience Scale (BRS) and Modeling Distinctiveness of Resilience from Depression and Stress. Psychology, 9, 1828-1857. https://doi.org/10.4236/psych.2018.97107
Kyriazos, T. A., Stalikas, A., Prassa, K., Yotsidi, V., Galanakis, M., & Pezirkianidis, C. (2018). Validation of the Flourishing Scale (FS), Greek Version and Evaluation of Two Well-Being Models. Psychology, 9, 1789-1813. https://doi.org/10.4236/psych.2018.97105
Little, J. (2013). Multilevel Confirmatory Ordinal Factor Analysis of the Life Skills Profile-16. Psychological Assessment, 25, 810-825. https://doi.org/10.1037/a0032574
Little, T. D., Schnabel, K. U., & Baumert, J. (2000). Modeling Longitudinal and Multilevel Data: Practical Issues, Applied Approaches, and Scientific Examples. Mahwah, NJ: Lawrence Erlbaum Associates Publishers. https://doi.org/10.4324/9781410601940
Loehlin, J. C., & Beaujean, A. A. (2017). Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis. New York, NY: Taylor & Francis.
Maas, C. J. M., & Hox, J. J. (2005). Sufficient Sample Sizes for Multilevel Modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 85-91. https://doi.org/10.1027/1614-2241.1.3.85
MacCallum, R. C., & Kim, C. (2000). Modeling Multivariate Change. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling Longitudinal and Multilevel Data: Practical Issues, Applied Approaches, and Scientific Examples (pp. 51-68). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
McArdle, J. J., & Hamagami, F. (1996). Multilevel Models from a Multiple Group Structural Equation Perspective. In G. A. Marcoulides, & R. E. Schumaker (Eds.), Advanced Structural Equation Modeling (pp. 89-124). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Mehta, P. D., & Neale, M. C. (2005). People Are Variables Too: Multilevel Structural Equation Modeling. Psychological Methods, 10, 259-284. https://doi.org/10.1037/1082-989X.10.3.259
Muthén, B. (2004). Latent Variable Analysis: Growth Mixture Modeling and Related Techniques for Longitudinal Data. In D. Kaplan (Ed.), Handbook of Quantitative Methodology for the Social Sciences (pp. 345-368). Newbury Park, CA: Sage.
Muthén, B. O. (1989). Latent Variable Modeling in Heterogenous Populations. Psychometrika, 54, 557-585. https://doi.org/10.1007/BF02296397
Muthén, B. O. (1990). Mean and Covariance Structure Analysis of Hierarchical Data. Paper Presented at the Psychometric Society Meeting, Princeton, June 1990, UCLA Statistics Series 62.
Muthén, B. O. (1991). Multilevel Factor Analysis of Class and Student Achievement Components. Journal of Educational Measurement, 28, 338-354. https://doi.org/10.1111/j.1745-3984.1991.tb00363.x
Muthén, B. O. (1994). Multilevel Covariance Structure Analysis. Sociological Methods and Research, 22, 376-398. https://doi.org/10.1177/0049124194022003006
Muthén, B. O. (1997). Latent Variable Modeling of Longitudinal and Multilevel Data. In A. E. Raftery (Ed.), Sociological Methodology 1997 (pp. 453-481). Washington DC: American Sociological Association. https://doi.org/10.1111/1467-9531.271034
Muthén, B. O., & Satorra, A. (1995). Complex Sample Data in Structural Equation Modeling. In P. Marsden (Ed.), Sociological Methodology 1995 (pp. 216-316). Boston, MA: Blackwell. https://doi.org/10.2307/271070
Muthén, B., Khoo, S.-T., Francis, D. J., & Boscardin, C. K. (2003). Analysis of Reading Skills Development from Kindergarten through First Grade: An Application of Growth Mixture Modeling to Sequential Processes. In S. P. Reise, & N. Duan (Eds.), Multilevel Modeling: Methodological Advances, Issues, and Applications (pp. 71-89). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus User’s Guide (Seventh ed.). Los Angeles, CA: Muthén & Muthén.
Nezlek, J. B. (2011). Multilevel Modeling for Social and Personality Psychology. London: Sage. https://doi.org/10.4135/9781446287996
Raftery, A. E. (1993). Bayesian Model Selection in Structural Equation Models. In K. A. Bollen, & J. S. Long (Eds.), Testing Structural Equation Models (pp. 163-180). Newbury Park, CA: Sage.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods (2nd ed.). Newbury Park, CA: Sage.
Reise, S. P., & Duan, N. (2003). Multilevel Modeling: Methodological Advances, Issues, and Applications. Mahwah, NJ: Lawrence Erlbaum Associates Publishers. https://doi.org/10.4324/9781410606747
Rigdon, E. (1998). Structural Equation Models. In G. Marcoulides (Ed.), Modern Methods for Business Research (pp. 251-294). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Ryu, E., & West, S. G. (2009). Level-Specific Evaluation of Model Fit in Multilevel Structural Equation Modeling. Structural Equation Modeling, 16, 583-601. https://doi.org/10.1080/10705510903203466
Satorra, A., & Bentler, P. M. (2010). Ensuring Positiveness of the Scaled Difference Chi-Square Test Statistic. Psychometrika, 75, 243. https://doi.org/10.1007/s11336-009-9135-y
Schumacker, R. E., & Lomax, R. G. (2016). A Beginner’s Guide to Structural Equation Modeling (4th ed.). New York: Routledge.
Schwartz, G. (1978). Estimating the Dimension of a Model. Annals of Statistics, 6, 461-464. https://doi.org/10.1214/aos/1176344136
Selig, J. P., Card, N. A., & Little, T. D. (2008). Latent Variable Structural Equation Modelling in Cross-Cultural Research: Multigroup and Multilevel Approaches. In F. J. R. van de Vijver, D. A. van Hemert, & Y. H. Poortinga (Eds.), Multilevel Analysis of Individuals and Cultures (pp. 93-119). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Singer, J. D., & Willett, J. B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. New York: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195152968.001.0001
Snijders, T., & Bosker, R. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Newbury Park, CA: Sage.
Stapleton, L. M. (2013). Incorporating Sampling Weights into Single- and Multi-Level Models. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), A Handbook of International Large-Scale Assessment (pp. 353-388). London: Chapman Hall/CRC Press.
Steiger, J. H., & Lind, J. C. (1980). Statistically Based Tests for the Number of Common Factors. Paper Presented at the Psychometric Society Annual Meeting, Iowa City, IA.
Tabachnick, B., & Fidell, L. (2013). Using Multivariate Statistics. Boston, MA: Pearson Education Inc.
Tucker, L. R., & Lewis, C. (1973). A Reliability Coefficient for Maximum Likelihood Factor Analysis. Psychometrika, 38, 1-10. https://doi.org/10.1007/BF02291170
van Buuren, S. (2011). Multiple Imputation of Multilevel Data. In J. J. Hox, & J. K. Roberts (Eds.), Handbook of Advanced Multilevel Analysis (pp. 173-196). New York: Routledge.
Yuan, K.-H., & Bentler, P. M. (2002). On Normal Theory Based Inference for Multilevel Models with Distributional Violations. Psychometrika, 67, 539-562. https://doi.org/10.1007/BF02295130
Yuan, K.-H., & Bentler, P. M. (2004). On the Asymptotic Distributions of Two Statistics for Two-Level Covariance Structure Models within the Class of Elliptical Distributions. Psychometrika, 69, 437-457. https://doi.org/10.1007/BF02295645
Yuan, K.-H., & Hayashi, K. (2005). On Muthen’s Maximum Likelihood for Two-Level Covariance Structure Models. Psychometrika, 70, 147-167. https://doi.org/10.1007/s11336-003-1070-8