Inverse Probability Weighting Estimation in Completely Randomized Experiments

Abstract

In addition to treatment assignments and observed outcomes, covariate information is often available prior to randomization in completely randomized experiments that compare an active treatment versus control. The analysis of covariance (ANCOVA) method is commonly applied to adjust for baseline covariates in order to improve precision. We focus on making propensity score-based adjustment to covariates under the completely randomized design in a finite population of experimental units with two treatment groups. We study inverse probability weighting (IPW) estimation of the finite-population average treatment effect for a general class of working propensity score models, which includes generalized linear models for binary data. We provide randomization-based asymptotic analysis of the propensity score approach and explore the finite-population asymptotic behaviors of two IPW estimators of the average treatment effect. We identify a condition under which propensity score-based covariate adjustment is asymptotically equivalent to an ANCOVA-based covariate adjustment and improves precision compared with a simple unadjusted comparison between treatment and control arms. In particular, when the working propensity score is fitted by a generalized linear model for binary data with an intercept term, the asymptotic variance of the IPW estimators is the same for any link function, including identity link, logit link, probit link, and complementary log-log link. We demonstrate these methods using an HIV clinical trial and a post-traumatic stress disorder study. Finally, we present a simulation study comparing the finite-sample performance of IPW and other methods for both continuous and binary outcomes. Supplementary materials for this article are available online.

KEYWORDS:

• Analysis of covariance
• Average treatment effect
• Covariate adjustment
• Generalized linear model
• Propensity score
• Regression

Supplementary Materials

This supplementary material contains theorems, their proofs, formulas for variance estimation, and Tables 5–8 for simulation results.

Acknowledgments

I am grateful to the editor, an associate editor, and two reviewers for providing two important references and for their helpful comments that have greatly improved my original submission.