Measuring Item Influence for Diagnostic Classification Models

ABSTRACT

Diagnostic classification models (DCMs) are psychometric models that provide probabilistic classifications of examinees on a set of discrete latent attributes. When analyzing or constructing assessments scored by DCMs, understanding how each item influences attribute classifications can clarify the meaning of the measured constructs, facilitate appropriate construct representation, and identify items contributing minimal utility. In cases of short assessments, common in the DCM literature, item influence becomes paramount as individual items can have a disproportionate impact on, or entirely determine, classification. This study proposes four indices to quantify item influence and distinguishes them from other available item and test measures. We use simulation methods to evaluate and provide guidelines for interpreting each index, followed by a real data application to illustrate their use in practice. We discuss theoretical considerations regarding when influence presents a psychometric concern and other practical concerns such as how the indices function when reducing influence imbalance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplemental material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10627197.2023.2244411

Additional information

Funding

This work was supported by the National Science Foundation, Division of Social and Economic Sciences under Grant No. 1921373.