The goal of this work was: 1) to present the assumptions for use of the estimators used in CFA/SEM and EFA, and their advantages/disadvantages; 2) to highlight that the variables were treated either as continuous or as ordinal categorical during the estimation process should be consistently treated for the rest of the study analyses (keeping a consistent level of measurement). Two estimator groups exist 1) Maximum Likelihood and 2) Least Squares. Robust alternatives exist in both groups. Desirable estimator properties include consistency, non-biasness, efficiency, scale freeness, and scale invariance. Scholars propose selecting an estimator considering: 1) measurement level; 2) non-normality; 3) model type. Not considering these could affect parameter significance, chi-square tests, and model fit. A plausible under-highlighted issue is the impact of the level of measurement implied during the estimation process on the level of measurement assumed for the same variables across the study. E.g., when an ordinal categorical level is assumed using categorical estimators, the same variables should be treated as ordinal categorical for the rest of the study: 1) using ordinal reliability; 2) omitting means; 3) using tetrachoric/polychoric correlation for the nomological network. Therefore, the selection of an estimator impacts all the analytic strategies of the study.
KeywordsEstimatorsEstimationDiscrepancy FunctionsCFASEMEFAAssumptions of UseLevel of MeasurementContinuous DataOrdinal Categorical Data
Asparouhov, T., & Muthén, B. (2005). Multivariate Statistical Modeling with Survey Data. In The 2005 FCSM Research Conference (pp. 14-16).
Bandalos, D. L., & Gerstner, J. J. (2016). Using Factor Analysis in Test Construction. In K. Schweizer, & C. DiStefano (Eds.), Principles and Methods of Test Construction: Standards and Recent Advances (pp. 26-51). Hogrefe.
Beauducel, A., & Herzberg, P. Y. (2006). On the Performance of Maximum Likelihood versus Mean and Variance Adjusted Weighted Least Squares Estimation in CFA. Structural Equation Modeling, 13, 186-203. https://doi.org/10.1207/s15328007sem1302_2
Beaujean, A. A. (2014). Latent Variable Modeling Using R: A Step-by-Step Guide. Routledge. https://doi.org/10.4324/9781315869780
Bentler, P. M. (2005). EQS 6 Structural Equations Program Manual. Multivariate Software.
Bentler, P. M., & Chou, C.-P. (1987). Practical Issues in Structural Modeling. Sociological Methods & Research, 16, 78-117. https://doi.org/10.1177/0049124187016001004
Bentler, P. M., & Yuan, K-H. (1999). Structural Equation Modeling with Small Samples: Test Statistics. Multivariate Behavioral Research, 34, 181-197. https://doi.org/10.1207/S15327906Mb340203
Bollen, K. A. (1989). Introduction to Structural Equation Models with Latent Variables. Wiley. https://doi.org/10.1002/9781118619179
Bollen, K. A. (2012). Instrumental Variables in Sociology and the Social Sciences. Annual Review of Sociology, 38, 37-72. https://doi.org/10.1146/annurev-soc-081309-150141
Bollen, K. A., & Barb, K. H. (1981). Pearson’s R and Coarsely Categorized Measures. American Sociological Review, 46, 232-239. https://doi.org/10.2307/2094981
Bollen, K. A., & Stine, R. A. (1993). Bootstrapping Goodness-of-Fit Measures in Structural Equation Models. In K. A. Bollen, & J. S. Long (Eds.), Testing Structural Equation Models (pp. 111-135). Sage.
Bollen, K. A., Kirby, J. B., Curran, P. J., Paxton, P. M., & Chen, F. (2007). Latent Variable Models under Misspecification: Two-Stage Least Squares (2SLS) and Maximum Likelihood (ML) Estimators. Sociological Methods and Research, 36, 48-86. https://doi.org/10.1177/0049124107301947
Briggs, N. E., & MacCallum, R. C. (2003). Recovery of Weak Common Factors by Maximum Likelihood and Ordinary Least Squares Estimation. Multivariate Behavioral Research, 38, 25-56. https://doi.org/10.1207/S15327906MBR3801_2
Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research (2nd ed.). Guilford.
Browne, M. W. (1974). Generalized Least Squares Estimators in the Analysis of Covariance Structures. South African Statistical Journal, 8, 1-24. https://doi.org/10.1002/j.2333-8504.1973.tb00197.x
Browne, M. W. (1984). Asymptotically Distribution-Free Methods for the Analysis of Covariance Structures. British Journal of Mathematical and Statistical Psychology, 37, 62-83. https://doi.org/10.1111/j.2044-8317.1984.tb00789.x
Byrne, B. M. (2012). Structural Equation Modeling with Mplus. Routledge. https://doi.org/10.4324/9780203807644
Carroll, J. B. (1993). Human Cognitive Abilities: A Survey of Factor-Analytic Studies. Cambridge University Press. https://doi.org/10.1017/CBO9780511571312
Chou, C. P., & Bentler, P. M. (1995). Estimates and Tests in Structural Equation Modeling.
Coenders, G., Satorra, A., & Saris, W. E. (1997). Alternative Approaches to Structural Modeling of Ordinal Data: A Monte Carlo Study. Structural Equation Modeling, 4, 261-282. https://doi.org/10.1080/10705519709540077
Costello, A. B., & Osborne, J. (2005). Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis. Practical Assessment Research & Evaluation, 10, 1-9.
Curran, P. J., West, S. G., & Finch, J. F. (1996). The Robustness of Test Statistics to Nonnormality and Specification Error in Confirmatory Factor Analysis. Psychological Methods, 1, 16-29. https://doi.org/10.1037/1082-989X.1.1.16
Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory Factor Analysis. Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199734177.001.0001
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the Use of Exploratory Factor Analysis in Psychological Research. Psychological Methods, 4, 272-299. https://doi.org/10.1037/1082-989X.4.3.272
Fielle, N. (2016). Basics of Matrix Algebra for Statistics with R. Journal of Statistical Software, 71, 1-3.
Finch, J. F., West, S. G., & MacKinnon, D. P. (1997). Effects of Sample Size and Nonnormality on the Estimation of Mediated Effects in Latent Variable Models. Structural Equation Modeling, 4, 87-107. https://doi.org/10.1080/10705519709540063
Flora, D. B. (2018). Statistical Methods for the Social and Behavioural Sciences: A Model-Based Approach. Sage.
Flora, D. B., & Curran, P. J. (2004). An Empirical Evaluation of Alternative Methods of Estimation for Confirmatory Factor Analysis with Ordinal Data. Psychological Methods, 9, 466-491. https://doi.org/10.1037/1082-989X.9.4.466
Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation. Structural Equation Modeling, 16, 625-641. https://doi.org/10.1080/10705510903203573
Fouladi, R. T. (2000). Performance of Modified Test Statistics in Covariance and Correlation Structure Analysis under Conditions of Multivariate Nonnormality. Structural Equation Modeling: A Multidisciplinary Journal, 7, 356-410. https://doi.org/10.1207/S15328007SEM0703_2
Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2012). Estimating Ordinal Reliability for Likert-Type and Ordinal Item Response Data: A Conceptual, Empirical, and Practical Guide. Practical Assessment, Research & Evaluation, 17, Article No. 3.
Gana, K., & Broc, G. (2019). Structural Equation Modeling with Lavaan. Wiley. https://doi.org/10.1002/9781119579038
Gorsuch, R. L. (1983). Factor Analysis. Erlbaum.
Gorsuch, R. L. (2015). Factor Analysis (3rd ed.). Routledge. https://doi.org/10.4324/9781315735740
Hu, L., Bentler, P. M., & Kano, Y. (1992). Can Test Statistics in Covariance Structure Analysis Be Trusted? Psychological Bulletin, 112, 351-362. https://doi.org/10.1037/0033-2909.112.2.351
Jöreskog, K. G. (1983). Factor Analysis as an Error-in-Variables Model. In H. Wainer, & S. Messick (Eds.), Principles of Modern Psychological Measurement (pp. 185-196). Erlbaum.
Jöreskog, K. G. (1994). On the Estimation of Polychoric Correlations and Their Asymptotic Covariance Matrix. Psychometrika, 59, 381-389. https://doi.org/10.1007/BF02296131
Jöreskog, K. G. (2007). Factor Analysis and Its Extensions. In R. Cudeck, & R. C. MacCallum (Eds.), Factor Analysis at 100: Historical Developments and Future Directions (pp. 47-78). Erlbaum.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: User’s Reference Guide. Scientific Software International.
Kaiser, H. F., & Caffrey, J. (1965). Alpha Factor Analysis. Psychometrika, 30, 1-14. https://doi.org/10.1007/BF02289743
Kaplan, D. (2000). Structural Equation Modeling: Foundations and Extensions. Sage. https://doi.org/10.4135/9781412984256
Kim, J., & Mueller, C. W. (1978). Factor Analysis: Statistical Methods & Practical Issues. Sage.
Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling. Guilford Press.
Kline, R. B. (2020). Becoming a Behavioral Science Researcher. Guilford.
Kyriazos, T. (2017). Reliability of Psychometric Instruments. In M. Galanakis, C. Pezirkianidis, & A. Stalikas (Eds.), Basic Aspects of Psychometrics (pp. 85-126). Topos.
Kyriazos, T. A. (2018). Applied Psychometrics: The Application of CFA to Multitrait-Multimethod Matrices (CFA-MTMM). Psychology, 9, 2625-2648. https://doi.org/10.4236/psych.2018.912150
Kyriazos, T. A. (2019). Applied Psychometrics: The Modeling Possibilities of Multilevel Confirmatory Factor Analysis (MLV CFA). Psychology, 10, 777-798. https://doi.org/10.4236/psych.2019.106051
Kyriazos, T. A., & Stalikas, A. (2019a). Nicomachus-Positive Parenting (NPP): Development and Initial Validation of a Parenting Questionnaire within the Positive Psychology Framework. Psychology, 10, 2115-2165. https://doi.org/10.4236/psych.2019.1015136
Kyriazos, T. A., & Stalikas, A. (2019b). Alabama Parenting Questionnaire—Short Form (APQ-9): Evidencing Construct Validity with Factor Analysis, CFA MTMM and Measurement Invariance in a Greek Sample. Psychology, 10, 1790-1817. https://doi.org/10.4236/psych.2019.1012117
Lee, S. Y., Poon, W. Y., & Bentler, P. M. (1995). A Two-Stage Estimation of Structural Equation Models with Continuous and Polytomous Variables. British Journal of Mathematical and Statistical Psychology, 48, 339-358. https://doi.org/10.1111/j.2044-8317.1995.tb01067.x
Lee, S.-Y., & Song, X.-Y. (2004). Evaluation of the Bayesian and Maximum Likelihood Approaches in Analyzing Structural Equation Models with Small Sample Sizes. Multivariate Behavioral Research, 39, 653-686. https://doi.org/10.1207/s15327906mbr3904_4
Lei, P.-W., & Wu, Q. (2012). Estimation in Structural Equation Modeling. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 164-180). Guilford.
Li, C. (2016). Confirmatory Factor Analysis with Ordinal Data: Comparing Robust Maximum Likelihood and Diagonally Weighted Least Squares. Behavior Research Methods, 48, 936-949. https://doi.org/10.3758/s13428-015-0619-7
Little, R. J. A., & Rubin, D. B. (1987). Statistical Analysis with Missing Data. Wiley.
Loehlin, J. C., & Beaujean, A. A. (2017). Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis (5th ed.). Routledge. https://doi.org/10.4324/9781315643199
MacCallum, R. C. (2009). Factor Analysis. In R. E. Millsap, & A. Maydeu-Olivares (Eds.), The Sage Handbook of Quantitative Methods in Psychology (pp. 123-147). Sage Publications Ltd. https://doi.org/10.4135/9780857020994.n6
MacCallum, R. C., Browne, M. W., & Cai, L. (2007). Factor Analysis Models as Approximations. In R. Cudeck, & R. C. MacCallum (Eds.), Factor Analysis at 100: Historical Developments and Future Directions (pp. 153-175). Erlbaum.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power Analysis and Determination of Sample Size for Covariance Structure Modeling. Psychological Methods, 1, 130-149. https://doi.org/10.1037/1082-989X.1.2.130
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler’s (1999) Findings. Structural Equation Modeling, 11, 320-341. https://doi.org/10.1207/s15328007sem1103_2
Matsunaga, M. (2010). How to Factor-Analyze Your Data Right: Do’s, Don’ts, and How-to’s. International Journal of Psychological Research, 3, 97-110. https://doi.org/10.21500/20112084.854
Micceri, T. (1989). The Unicorn, the Normal Curve, and Other Improbable Creatures. Psychological Bulletin, 105, 156-166. https://doi.org/10.1037/0033-2909.105.1.156
Moustaki, I. (2001). A Review of Exploratory Factor Analysis for Ordinal Categorical Data. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural Equation Modeling: Present and Future (pp. 461-480). Scientific Software.
Mulaik, S. A. (2009). Foundations of Factor Analysis. CRC press.
Muthén, B., & Kaplan, D. (1985). A Comparison of Some Methodologies for the Factor Analysis of Non-Normal Likert Variables. British Journal of Mathematical and Statistical Psychology, 38, 171-189.
Muthén, B., & Kaplan, D. (1992). A Comparison of Some Methodologies for the Factor Analysis of Non-Normal Likert Variables: A Note on the Size of the Model. British Journal of Mathematical and Statistical Psychology, 45, 19-30. https://doi.org/10.1111/j.2044-8317.1992.tb00975.x
Muthén, L. K., & Muthén, B. O (2006). Mplus User’s Guide (4th ed.). Muthén & Muthén.
Osborne, J. W., & Banjanovic, E. S. (2016). Exploratory Factor Analysis with SAS. SAS Institute.
R Development Core Team (2022). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.
Rao, C. R. (1955). Estimation and Tests of Significance in Factor Analysis. Psychometrika, 20, 93-111. https://doi.org/10.1007/BF02288983
Raykov, T. (2012). Scale Construction and Development Using Structural Equation Modeling. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 472-492). Guildford. https://doi.org/10.4324/9780203930687-1
Raykov, T., & Marcoulides, G. A. (2006). A First Course in Structural Equation Modeling. Erlbaum.
Raykov, T., & Widaman, K. F. (1995). Issues in Structural Equation Modeling Research. Structural Equation Modeling, 2, 289-318. https://doi.org/10.1080/10705519509540017
Revelle, W. (2016). A Review of Linear Algebra: Applications in R: Notes for Courses in Psychometric Theory and Latent Variable Modeling to Accompany Psychometric Theory with Applications in R. Northwestern University.
Revelle, W. (2022). Package “Psych”: Procedures for Psychological, Psychometric, and Personality Research. https://personality-project.org/r/psych
Rigdon, E. E. (1998). Structural Equation Modeling. In G. A. Marcoulides (Ed.), Modern Methods for Business Research (pp. 251-294). Erlbaum.
Rossel, Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48, 1-36. http://www.jstatsoft.org/v48/i02 https://doi.org/10.18637/jss.v048.i02
Russell, D. W. (2002). In Search of Underlying Dimensions: The Use (and Abuse) of Factor Analysis in Personality and Social Psychology Bulletin. Personality and Social Psychology Bulletin, 28, 1629-1646. https://doi.org/10.1177/014616702237645
Satorra, A., & Bentler, P. M. (1988). Scaling Corrections for Chi-Square Statistics in Covariance Structure Analysis. In American Statistical Association 1988 Proceedings of the Business and Economics Section (pp. 308-313). American Statistical Association.
Schumacker, R. E., & Beyerlein, S. T. (2000). Confirmatory Factor Analysis with Different Correlation Types and Estimation Methods. Structural Equation Modeling, 7, 629-636. https://doi.org/10.1207/S15328007SEM0704_6
Schumacker, R. E., & Lomax, R. G. (2016). A Beginner’s Guide to Structural Equation Modeling (4th ed.). Routledge. https://doi.org/10.4324/9781315749105
Skinner, C. J. (1989). Domain Means, Regression and Multivariate Analysis. In C. J. Skinner, D. Holt, & T. M. F. Smith (Eds.), Analysis of Complex Surveys (pp. 59-87). Wiley.
Stalikas, A., & Kyriazos, T. (2019). Research Methods and Statistics Using R. Topos.
Stalikas, A., Kyriazos, T. A., Yotsidi, V., & Prassa, K. (2018). Using Bifactor EFA, Bifactor CFA and Exploratory Structural Equation Modeling to Validate Factor Structure of the Meaning in Life Questionnaire, Greek Version. Psychology, 9, 348-371.
Tabachnick, B. G., & Fidell, L. S. (2012). Using Multivariate Statistics. 6h Edition, Person Education.
Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics. Allyn & Bacon.
Thompson, B. (2004). Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. American Psychological Association. https://doi.org/10.1037/10694-000
Thompson, B., & Daniel, L. G. (1996). Factor Analytic Evidence for the Construct Validity of Scores: A Historical Overview and Some Guidelines. Educational and Psychological Measurement, 56, 197-208. https://doi.org/10.1177/0013164496056002001
Watkins, M. W. (2021). A Step-by-Step Guide to Exploratory Factor Analysis with R and Rstudio. Routledge. https://doi.org/10.4324/9781003120001
Watson, J. C. (2017). Establishing Evidence for Internal Structure Using Exploratory Factor Analysis. Measurement and Evaluation in Counseling and Development, 50, 232-238. https://doi.org/10.1080/07481756.2017.1336931
West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural Equation Models with Nonnormal Variables: Problems and Remedies. In R. H. Hoyle (Ed.), Structural Equation Modeling: Concepts, Issues, and Applications (pp. 56-75). Sage.
Whittaker, T. A., & Schumacker, R. E. (2022). A Beginner’s Guide to Structural Equation Modeling (5th ed.). Routledge. https://doi.org/10.4324/9781003044017
Widaman, K. F. (2012). Exploratory Factor Analysis and Confirmatory Factor Analysis. In H. Cooper (Ed.), APA Handbook of Research Methods in Psychology: Data Analysis and Research Publication (Vol. 3, pp. 361-389). American Psychological Association. https://doi.org/10.1037/13621-018
Yuan, K. H., & Bentler, P. M. (2000). Three Likelihood-Based Methods for Mean and Covariance Structure Analysis with Nonnormal Missing Data. In Sociological Methodology (pp. 165-200). American Sociological Association. https://doi.org/10.1111/0081-1750.00078
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal Versions of Coefficients Alpha and Theta for Likert Rating Scales. Journal of Modern Applied Statistical Methods, 6, 21-29. https://doi.org/10.22237/jmasm/1177992180