Analytic Approaches to Handle Missing Data in Simple Matrix Sampling Planned Missing Designs

Abstract

Simple matrix sampling planned missing (SMS PD) design, introduce missing data patterns that lead to covariances between variables that are not jointly observed, and create difficulties for analyses other than mean and variance estimations. Based on prior research, we adopted a new multigroup confirmatory factor analysis (CFA) approach to handle missing data in such designs, in comparison to a regular CFA with full information maximum likelihood estimator. In Study 1, we tested the two approaches in 36 scenarios (4 sample sizes $\times$ 3 inter-item correlations $\times$ 3 numbers of x-set items) given a total of 20 items. We found that, the multigroup CFA approach performed with acceptable convergence rates, power to recover population values, acceptable standard errors and model fit in certain scenarios by larger sample size, higher bivariate correlation, and more items in the x-set. We found a few scenarios where regular CFA with FIML performed well. These findings suggested that the approaches can be implemented to handle the special missing data introduced by the SMS PM designs, and, thereby, enhance the utility of SMS PM data. In Study 2, we applied the multigroup CFA approach in real-world data to demonstrate the feasibility and analytic value of this approach.

Keywords:

• Full information maximum likelihood
• missing data
• multigroup confirmatory factor analysis
• planned missing designs
• simple matrix sampling

Acknowledgments

Special thanks to all participants of the study and graduate assistants who provided help with data collection. Thanks to Dr. Qiangguo Ren for his guidance and help with R coding.

Disclosure statement

The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Notes

1 Note that we set that the non-x sets to have the same number of items, i.e., $s=t=u,$ such that the three test forms are equivalent in length.

2 Using the default coverage in Mplus, none of the regular CFA with ML estimator can be performed.

Additional information

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A160335.