Analyzing Multivariate Generalizability Theory Designs within Structural Equation Modeling Frameworks

Abstract

We demonstrate how to analyze complete multivariate generalizability theory (GT) designs within structural equation modeling frameworks that encompass both individual subscale scores and composites formed from those scores. Results from numerous analyses of observed scores obtained from respondents who completed the recently updated form of the Big Five Inventory (BFI-2) revealed that the lavaan SEM package in R produced results virtually identical to those obtained from the mGENOVA package, which historically has served as the gold standard for conducting multivariate GT analyses. We further extended lavaan analyses beyond what mGENOVA allows to produce Monte Carlo based confidence intervals for key GT parameters and correct score consistency and correlational indices for effects of scale coarseness characteristic of binary and ordinal data. Our comprehensive online Supplemental Material includes code for performing all illustrated analyses using lavaan and mGENOVA.

Keywords:

• Generalizability theory
• multivariate analysis
• R programming
• structural equation modeling

Acknowledgements

The authors thank the Iowa Measurement Research Foundation for providing research assistantship support for this project.