Scale Reliability Evaluation Using Bayesian Analysis: A Latent Variable Modeling Procedure

ABSTRACT

An application of Bayesian factor analysis for evaluation of scale reliability is discussed, which is developed within the framework of latent variable modeling. The method permits direct point and interval estimation of the reliability coefficient of multiple-component measuring instruments using Bayesian inference. The approach allows also point and interval estimation of the population discrepancy between the popular coefficient alpha and instrument reliability. The procedure is readily applied in empirical measurement research employing widely available statistical software. The outlined method is illustrated using numerical data.

KEYWORDS:

• Bayesian statistics
• coefficient alpha
• credible interval
• latent variable modeling
• multi-component measuring instrument
• posterior distribution
• scale reliability

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. As can be found by replicating the results reported in this section and examining the pertinent output section, and as implied by the used Mplus source code in Appendix 2, we use default priors that are: (a) normal with mean 0 and infinite variance for factor loadings and mean intercepts, and (b) Inverse Gamma distributions with parameters −1 and 0 for residual variances (see also discussion and conclusion section as well as Footnote 3 below for motivation of their choice). Thereby, for the R-hat values (proportional scale reduction factors, abbr. PSRFs) also the default software convergence cutoff of .05 was utilized. All parameters were found thereby to have converged (with all parameters’ PSRFs being smaller than 1.05), as seen in the last section of the output produced thereby (see also B. Muthén et al., 2016; Gelman et al., 2013; and the penultimate output section for the used priors when fitting the single-factor model of relevance in the main text).

2. In the reported analyses in this paragraph of the main text, and with the software employed, (i) half of the requested iterations – that is, 100,000 - were utilized as burn-in iterations within each of the chains (which were 2 as used in the section; see Appendix 2). For each parameter, like mentioned in Footnote 1, (ii) convergence was assessed using the Gelman-Rubin convergence criterion based on the PSRF indices, and (iii) this process was aided by examining the trace plots of the posterior draws in the chains as well as the auto-correlation plots. (All these plots and following results are obtained when using the source codes in Appendices 1 and 2.) In that convergence assessment process, (iv) both chains were accordingly found to mix well; for this reason, (v) the default of 1 iteration was used for thinning (i.e., each iteration was used for the analytic purposes after the burn-in iterations). No less importantly, as indicated in the main text, (vi) the overall model fit was assessed by posterior predictive checking (PPC) where the posterior predictive distribution is compared to the observed data, and using (vii) the observed-replicated chi-square differences confidence interval. The p-value mentioned in the above point (vi) (also referred to as PPP-value), was computed by the software based on the likelihood-ratio chi-square statistic. Further details are found in L. K. Muthén and Muthén (2023, ch. 9; see also Gelman et al., 2013; additional specific model testing details are found in B. Muthén et al., 2016, pp. 400–401).

3. We elected to use non-informative priors in the illustration section owing to there being in general infinitely many possible informative priors for a given parameter. Hence, an appropriate choice among them is better carried out by substantive experts in a domain of application collaborating with Bayesian statisticians preferably after a thorough elicitation of parameter prior distribution, which is to be grounded in extant research. Moreover, we employed non-informative priors there, in order not to be suggestive of particular priors to possible users of the outlined procedure (Note 2 to Appendix 2 outlines how to utilize alternatively informative priors with the used software).