Multiple Group Analysis in Multilevel Data Across Within-Level Groups: A Comparison of Multilevel Factor Mixture Modeling and Multilevel Multiple-Indicators Multiple-Causes Modeling

Abstract

The purpose of this two-part study is to evaluate methods for multiple group analysis when the comparison group is at the within level with multilevel data, using a multilevel factor mixture model (ML FMM) and a multilevel multiple-indicators multiple-causes (ML MIMIC) model. The performance of these methods was evaluated integrally by a series of procedures testing weak and strong invariance models and the latent group mean differences testing after holding for factorial invariance. Two Monte Carlo simulation studies were conducted under the following conditions: number of clusters, cluster size, and the design type in groups. A multilevel one-factor confirmatory factor analysis (CFA) model as a research model in Study 1 was investigated to compare the results under different conditions with those of previous studies. A multilevel two-factor CFA model as a research model in Study 2 was evaluated by fitting alternative models that can be applied when the model is complicated. The results indicated that the two methods were reasonable in multilevel multiple groups analysis across within-level groups. However, pros and cons were found between the two methods. In the multilevel one-factor CFA model, ML MIMIC model was slightly better when the sample size is small. In the multilevel complex model, two alternative models of ML FMM were recommended because the weak invariance testing of ML MIMIC was considerably time-consuming. Finally, it was shown that information criteria, which are criteria for determining whether factorial invariance is established, need to be applied differently according to the sample size conditions. Guidelines for this situation are provided.