Coefficient alpha and reliability of communication science measurement scales: A note on Hayes and Coutts

ABSTRACT

This note intends to complement the recent discussion in Hayes and Coutts (2020) by focusing on (i) the loading equality condition for the population identity of coefficient alpha and reliability of multiple-indicator measurement scales, as well as (ii) the potential utility of alpha when this condition is not satisfied. We show that the alpha and reliability coefficients can be very close at the population level in certain cases of loading inequality. In addition, we point out that in any studied population the identity of alpha and scale reliability (coefficient omega) is an improbable event. We discuss implications for communication and behavioral research with large samples, which are becoming increasingly widely used in large-scale and nationally representative studies. Findings of the article are then illustrated using numerical data. We conclude with proposed recommendations for the use of coefficients alpha and omega by communication and behavioral scientists concerned with evaluating scale reliability.

KEYWORDS:

• coefficient alpha
• multi-item measuring scale
• scale reliability
• probability
• population

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. In the setting considered here (see Equations (1) and their immediately preceding and succeeding discussion), the single-factor model is empirically indistinguishable from the popular congeneric test model (Jöreskog, 1971), which is often used alternatively as a basis of reliability related discussions in the literature. The reason is that the two models have the same implications for means, covariance and correlation matrices as well as third- and fourth-order moments. (The set of these statistics represent all moments of relevance for estimating covariance structure models with contemporary, widely used model fitting means; e.g., Bollen, 1989.)

2. One may on occasion find in various reliability-related treatments and accounts references to coefficient omega as “McDonald’s omega,” “Jöreskog’s rho,” or “Raykov’s rho.” Unlike those references, throughout this note we refrain from attaching a personal name to omega. Instead, we emphasize the relevant population definitions of both coefficients alpha and omega, which are of key relevance to this article (see, e.g., Equations (2) and (3); Raykov and Marcoulides (2011), Raykov (2012), Raykov etal. (2016).

3. We stress that all results reported in the illustration section are directly replicated by using the two command files in Appendix 1, when employing the seed utilized in the first of them. Thereby, as can be readily verified in this way, none of the simulated 10,000 data sets yields an inadmissible solution, and for all of them the associated numerical optimization procedure converges. In addition, 5% (rounded off) of these data sets lead to model rejection, as judged by the resulting chi-square value (specifically, exceeding the theoretical chi-square distribution cutoff of 5.99 at the commonly used .05 significance level). This rejection rate is identical to what would be expected by chance alone – viz. 5% - under the SFM fitted. With the relatively high replication “sample size” of 1000, 97% of the respective 10,000 root mean square error of approximation (RMSEA) index estimates are found to range between 0 and .05, and 99% under .06, indicating plausible fit as well (Hu & Bentler, 1999). Relatedly, all but one of the lower endpoints of the RMSEA 90%-confidence intervals are under .06 (with the highest being under .07), corroborating further this model’s plausibility interpretation with respect to the 10,000 replication data sets that the illustration section is based on.

Additional information

Notes on contributors

Tenko Raykov

Tenko Raykov is Professor of Quantitative Methods at the Graduate Program in Measurement and Quantitative Methods of Michigan State University. His specialization areas include latent variable and structural equation modeling, scale construction and development, multilevel modeling, multivariate statistics, missing data analysis, longitudinal data modeling, item response theory.

George A. Marcoulides

George A. Marcoulides is Professor of Statistics at the School of Business of Texas A & M University. His specialization areas include latent variable and structural equation modeling, psychometrics, item response theory, multivariate statistics, scale construction and development, multilevel modeling.

Natalja Menold

Natalja Menold is Professor of Empirical Social Research at the Institute of Sociology of the Technical University Dresden, Germany. Her specialization areas include latent variable and structural equation modeling, psychometrics, Bayesian statistics, applied measurement procedures, alignment, and the study of measurement invariance and equivalence.