Propensity Score Weighting with Missing Data on Covariates and Clustered Data Structure

Abstract

Propensity score (PS) analyses are increasingly popular in behavioral sciences. Two issues often add complexities to PS analyses, including missing data in observed covariates and clustered data structure. In previous research, methods for conducting PS analyses with considering either issue alone were examined. In practice, the two issues often co-occur; but the performance of methods for PS analyses in the presence of both issues has not been evaluated previously. In this study, we consider PS weighting analysis when data are clustered and observed covariates have missing values. A simulation study is conducted to evaluate the performance of different missing data handling methods (complete-case, single-level imputation, or multilevel imputation) combined with different multilevel PS weighting methods (fixed- or random-effects PS models, inverse-propensity-weighting or the clustered weighting, weighted single-level or multilevel outcome models). The results suggest that the bias in average treatment effect estimation can be reduced, by better accounting for clustering in both the missing data handling stage (such as with the multilevel imputation) and the PS analysis stage (such as with the fixed-effects PS model, clustered weighting, and weighted multilevel outcome model). A real-data example is provided for illustration.

Keywords:

• Propensity score weighting
• multilevel data
• missing data
• multiple imputation
• causal inference

Article information

Conflict of interest disclosures: The author signed a form for disclosure of potential conflicts of interest. The author did not report any financial or other conflicts of interest in relation to the work described.

Ethical principles: The author affirms having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This research received no external funding.

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The author would like to thank the anonymous reviewers and the associate editor for their comments on prior versions of this manuscript, and Dr. Brian T. Keller for his feedback during early stages of this study. The ideas and opinions expressed herein are those of the author alone, and endorsement by the author’s institution is not intended and should not be inferred.

Notes

1 Several types of methods are not considered in our current study. For example, the separate missing pattern approach (D’Agostino et al., 2001; Rosenbaum & Rubin, 1983) was not considered, and it has been shown that this approach can be infeasible when some missing patterns have small sample sizes and can be inadequate for balancing the missing covariates in each missing pattern (Cham & West, 2016; Leyrat et al., 2019; Qu & Lipkovich, 2009). The general location approach (D’Agostino & Rubin, 2000) was not considered; it has been argued that this approach requires a stringent assumption (the propensity score depends on a covariate only if the covariate value is not missing) and is computationally intensive (Coffman et al., 2020; Seaman & White, 2014).