A systematic comparison of non-overlap metrics and visual analysis in single-case experimental design

Abstract

Visual analysis of graphed data is a prominent strategy employed by single-case researchers to document data stability and to ascertain whether a causal relation is demonstrated between the intervention and the dependent variable. Contemporary best practice also includes the use of non-parametric non-overlap metrics or parametric effect sizes to systematically quantify the magnitude of intervention effects. In the present study, 132 studies with 924 AB phase contrasts were analyzed using visual analysis and three non-overlap metrics. Sensitivity and specificity of Tau-U, improvement rate difference, and baseline-corrected Tau were tested using receiver operating characteristic analysis to detect the magnitude of intervention effects. Agreement with visual analysis was examined using kappa coefficient, and Pearson’s correlation coefficient. Each of the three non-overlap metrics performed moderately well in distinguishing effective from non-effective interventions and had a moderate to substantial level of agreement with visual analysis. Further research on the impact of confounding factors as well as the use of sensitive effect size metrics and visual aids to improve the reliability and accuracy of visual analysis is recommended.

Keywords:

• Non-overlap methods
• sensitivity
• single-case experimental designs
• specificity
• visual analysis

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the author(s).

Notes

¹ SPSS is a registered trademark of International Business Machines (IBM) Corporation, registered in many jurisdictions worldwide.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.