Unique Variable Analysis: A Network Psychometrics Method to Detect Local Dependence

Abstract

The local independence assumption states that variables are unrelated after conditioning on a latent variable. Common problems that arise from violations of this assumption include model misspecification, biased model parameters, and inaccurate estimates of internal structure. These problems are not limited to latent variable models but also apply to network psychometrics. This paper proposes a novel network psychometric approach to detect locally dependent pairs of variables using network modeling and a graph theory measure called weighted topological overlap (wTO). Using simulation, this approach is compared to contemporary local dependence detection methods such as exploratory structural equation modeling with standardized expected parameter change and a recently developed approach using partial correlations and a resampling procedure. Different approaches to determine local dependence using statistical significance and cutoff values are also compared. Continuous, polytomous (5-point Likert scale), and dichotomous (binary) data were generated with skew across a variety of conditions. Our results indicate that cutoff values work better than significance approaches. Overall, the network psychometrics approaches using wTO with graphical least absolute shrinkage and selector operator with extended Bayesian information criterion and wTO with Bayesian Gaussian graphical model were the best performing local dependence detection methods overall.

Keywords:

• Local dependence
• correlated residuals
• minor factors

Open Scholarship

This article has earned the Center for Open Science badges for Open Data and Open Materials through Open Practices Disclosure. The data and materials are openly accessible at https://doi.org/10.5281/zenodo.1212328 and https://doi.org/10.17605/osf.io/9w3jy. To obtain the author's disclosure form, please contact the Editor.

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: L.E.G. was supported by Grant 2018-2019-1D2-085 from the Fondo Nacional de Innovación y Desarrollo Científico y Tecnológico (FONDOCYT) of the Dominican Republic.

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgements: The authors did not preregister the study. All data, code, and materials can be found on the Open Science Framework: https://osf.io/9w3jy/. Alexander P. Christensen https://orcid.org/0000-0002-9798-7037, Luis Eduardo Garrido https://orcid.org/0000-0001-8932-6063, and Hudson Golino https://orcid.org/0000-0002-1601-1447. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institutions or the funding agency is not intended and should not be inferred.

Data availability statement

All data and R scripts can be found on the Open Science Framework: https://osf.io/9w3jy/.

The authors made the following contributions. Alexander P. Christensen: Conceptualization, Data Curation, Formal Analysis, Methodology, Software, Visualization, Writing - Original Draft, Writing - Review & Editing; Luis Eduardo Garrido: Conceptualization, Formal Analysis, Supervision, Writing - Review & Editing; Hudson Golino: Conceptualization, Formal Analysis, Data Curation, Supervision, Writing - Review & Editing.

Notes

1 In the psychometric network literature, it is more common to use the Ising model for dichotomous data (van Borkulo et al., 2014); however, to demonstrate the generalizability of our approach to detect local dependence, we use the EBICglasso with tetrachoric correlations rather than the Ising model.

2 Categorical factor analysis and structural equation modeling are alternative parameterizations of item response theory models (Muraki & Carlson, 1995). The main difference in these models is the estimation procedures rather than the models themselves (limited information estimation procedures for the former; full-information for the latter). We thank the anonymous reviewer for pointing out these similarities.

3 Recent research has demonstrated that Spearman’s correlation may actually be more appropriate for ordinal data when using the EBICglasso method (Isvoranu & Epskamp, 2021). We present the results of Spearman’s correlation in the Supplemental Information (SI 3). We find that although Spearman’s correlation performs about as well as Pearson’s and polychoric correlations, they do not fare as well as tetrachoric correlations in dichotomous data. For this reason, we present only the results of EBICglasso using Pearson’s, polychoric, and dichotomous correlations in the main text.