Performance of Model Fit and Selection Indices for Bayesian Piecewise Growth Modeling with Missing Data

Abstract

The Bayesian piecewise growth model (PGM) is a useful class of models for analyzing nonlinear change processes that consist of distinct growth phases. In applications of Bayesian PGMs, it is important to accurately capture growth trajectories and carefully consider knot placements. The presence of missing data is another challenge researchers commonly encounter. To address these issues, one could use model fit and selection indices to detect misspecified Bayesian PGMs, and should give care to the potential impact of missing data on model evaluation. Here we conducted a simulation study to examine the impact of model misspecification and missing data on the performance of Bayesian model fit and selection indices (PPP-value, BCFI, BTLI, BRMSEA, BIC, and DIC), with an additional focus on prior sensitivity. Results indicated that (a) increasing the degree of model misspecification and amount of missing data aggravated the performance of indices in detecting misfit, and (b) different prior specifications had negligible impact on model assessment. We provide practical guidelines for researchers to facilitate effective implementation of Bayesian PGMs.

Keywords:

• Bayesian piecewise growth
• missing data
• model fit
• model misspecification
• model selection

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 We provide two cautionary notes for our readers. First, when we use the term “Bayesian model selection indices”, we are referring to both the BIC and DIC, not just the BIC alone. Second, the computation of the BIC relies on the maximum likelihood estimates of the parameters. That said, the BIC cannot be considered a purely “Bayesian measure” given that its specification is not based on the posterior distribution (Gelman et al., 2014). However, we included the BIC in the current investigation because (1) it is provided as default in Mplus when using the Bayesian estimation framework, and (2) its performance was investigated in previous studies on Bayesian model evaluation (Depaoli et al., 2023; Winter & Depaoli, 2022a, 2022b).

2 As of Mplus version 8.4, missing values remain missing when computing the discrepancy function (see Asparouhov & Muthén, 2021). The replicated data generated at each iteration using the incomplete observed data mimics the missing data patterns. The same missing data patterns between the observed and replicated data ensure the comparability of both data under the null hypothesis that ${\mathcal{M}}_0$ is true. In addition, Asparouhov and Muthén (2021) pointed out that replicated data can be considered MAR under the assumption that the actual data is MAR. Therefore, comparability of the discrepancy function for the observed and replicated data is ensured.

3 Online Supplementary Materials are available at the Open Science Framework project (https://osf.io/agn4x/).

4 We highlight that this value is not a strict cutoff; nor should it be. We are not advocating for implementation of frequentist cutoffs for these Bayesian indices. Instead, the use of cutoff values is rather to facilitate the intuitive interpretation of results that represents the likely way the indices will be used and interpreted in practice. We take this approach in interpreting the other Bayesian model fit indices as well.