Overlap Violations in Clustered Observational Studies of Educational Interventions

References

• Altonji, J. G., Elder, T. E., & Taber, C. R. (2005). Selection on observed and unobserved variables: Assessing the effectiveness of Catholic schools. Journal of Political Economy, 113(1), 151–184. https://doi.org/10.1086/426036
   Web of Science ®Google Scholar
• Bryk, A. S., Lee, V. E., & Holland, P. B. (1993). Catholic schools and the common good. Basic Books.
   Google Scholar
• Cochran, W. G., & Chambers, S. P. (1965). The planning of observational studies of human populations. Journal of Royal Statistical Society, Series A, 128(2), 234–265. https://doi.org/10.2307/2344179
   Google Scholar
• Crump, R. K., Hotz, V. J., Imbens, G. W., & Mitnik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1), 187–199. https://doi.org/10.1093/biomet/asn055
   Web of Science ®Google Scholar
• D’Amour, A., Ding, P., Feller, A., Lei, L., & Sekhon, J. (2021). Overlap in observational studies with high-dimensional covariates. Journal of Econometrics, 221(2), 644–654. https://doi.org/10.1016/j.jeconom.2019.10.014
   Web of Science ®Google Scholar
• Dehejia, R., & Wahba, S. (1999). Causal effects in non-experimental studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association, 94(448), 1053–1062. https://doi.org/10.1080/01621459.1999.10473858
   Web of Science ®Google Scholar
• Grogger, J., & Neal, D. (2000). Further evidence on the effects of catholic secondary schooling. Brookings-Wharton Papers on Urban Affairs, 2000(1), 151–193. https://doi.org/10.1353/urb.2000.0006
   Google Scholar
• Hernán, M. A., & Robins, J. M. (2020). Causal inference: What if. Chapman& Hall/CRC.
   Google Scholar
• Hill, J., & Su, Y.-S. (2013). Assessing lack of common support in causal inference using Bayesian nonparametrics: Implications for evaluating the effect of breastfeeding on children’s cognitive outcomes. The Annals of Applied Statistics, 7(3), 1386–1420. https://doi.org/10.1214/13-AOAS630
   Web of Science ®Google Scholar
• Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15(3), 199–236. https://doi.org/10.1093/pan/mpl013
   Web of Science ®Google Scholar
• Imbens, G. W., & Rubin, D. B. (2015). Causal inference for statistics, social, and biomedical sciences: An introduction. Cambridge University Press.
   Google Scholar
• Keele, L. J., Lenard, M., & Page, L. (2021). Matching methods for clustered observational studies in education. Journal of Educational Effectiveness, 14(3), 696–725.
   Web of Science ®Google Scholar
• Keele, L., & Pimentel, S. (2016). matchMulti: Optimal multilevel matching using a network algorithm. r package version 1.1.5.
   Google Scholar
• Keele, L. J., & Zubizarreta, J. (2017). Optimal multilevel matching in clustered observational studies: A case study of the effectiveness of private schools under a large-scale voucher system. Journal of the American Statistical Association, 112(518), 547–560. https://doi.org/10.1080/01621459.2016.1240683
   Web of Science ®Google Scholar
• King, G., & Zeng, L. (2006). The dangers of extreme counterfactuals. Political Analysis, 14(2), 131–159. https://doi.org/10.1093/pan/mpj004
   Web of Science ®Google Scholar
• Kirksey, J. J., & Gottfried, M. A. (2021). Is there a Catholic school effect on kindergarteners’ absenteeism? Journal of Research on Educational Effectiveness, 14(3), 570–597. https://doi.org/10.1080/19345747.2021.1898707
   Web of Science ®Google Scholar
• Lee, V. E., & Bryk, A. S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62(3), 172–192. https://doi.org/10.2307/2112866
   Web of Science ®Google Scholar
• Mamdani, M., Sykora, K., Li, P., Normand, S.-L T., Streiner, D. L., Austin, P. C., Rochon, P. A., & Anderson, G. M. (2005). Reader’s guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ, 330(7497), 960–962. https://doi.org/10.1136/bmj.330.7497.960
   PubMedGoogle Scholar
• Manski, C. F. (2007). Identification for prediction and decision. Harvard University Press.
   Google Scholar
• Morgan, S. L. (2001). Counterfactuals, causal effect heterogeneity, and the Catholic school effect on learning. Sociology of Education, 74(4), 341–374. https://doi.org/10.2307/2673139
   Web of Science ®Google Scholar
• Morgan, S. L., & Harding, D. J. (2006). Matching estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods & Research, 35(1), 3–60. https://doi.org/10.1177/0049124106289164
   Web of Science ®Google Scholar
• Morgan, S. L., & Todd, J. J. (2008). A diagnostic routine for the detection of consequential heterogeneity of causal effects. Sociological Methodology, 38(1), 231–282. https://doi.org/10.1111/j.1467-9531.2008.00204.x
   Web of Science ®Google Scholar
• Neal, D. A. (1997). The effects of catholic secondary schooling on educational achievement. Journal of Labor Economics, 15(1, Part 1), 98–123. https://doi.org/10.1086/209848
   Web of Science ®Google Scholar
• Neyman, J., Dabrowska, D. M., & Terence, P. (1923). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5, 465–472.
   Google Scholar
• Page, L. C., Lenard, M., & Keele, L. (2020). The design of clustered observational studies in education. AERA Open, 6(3), 233285842095440. https://doi.org/10.1177/2332858420954401
   Web of Science ®Google Scholar
• Petersen, M. L., Porter, K. E., Gruber, S., Wang, Y., & van der Laan, M. J. (2012). Diagnosing and responding to violations in the positivity assumption. Statistical Methods in Medical Research, 21(1), 31–54. https://doi.org/10.1177/0962280210386207
   PubMed Web of Science ®Google Scholar
• Pimentel, S. D., Page, L. C., Lenard, M., & Keele, L. J. (2018). Optimal multilevel matching using network flows: An application to a summer reading intervention. Annals of Applied Statistics, 12, 1479–1505.
   Web of Science ®Google Scholar
• Raudenbush, S. W., & Bryk, A. (2002). Hierarchical linear models: Applications and data analysis methods. Sage.
   Google Scholar
• Reardon, S. F., Cheadle, J. E., & Robinson, J. P. (2009). The effect of Catholic schooling on math and reading development in kindergarten through fifth grade. Journal of Research on Educational Effectiveness, 2(1), 45–87. https://doi.org/10.1080/19345740802539267
   Google Scholar
• Rosenbaum, P. R. (1989). Optimal matching for observational studies. Journal of the American Statistical Association, 84(408), 1024–1032. https://doi.org/10.1080/01621459.1989.10478868
   Web of Science ®Google Scholar
• Rosenbaum, P. R. (2010). Design of observational studies. Springer-Verlag.
   Google Scholar
• Rosenbaum, P. R. (2012). Optimal matching of an optimally chosen subset in observational studies. Journal of Computational and Graphical Statistics, 21(1), 57–71. https://doi.org/10.1198/jcgs.2011.09219
   Web of Science ®Google Scholar
• Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350
   Web of Science ®Google Scholar
• Rubin, D. B. (1986). Which ifs have causal answers. Journal of the American Statistical Association, 81, 961–962.
   Web of Science ®Google Scholar
• Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21. https://doi.org/10.1214/09-STS313
   PubMed Web of Science ®Google Scholar
• Yang, S., & Ding, P. (2018). Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores. Biometrika, 105(2), 487–493. https://doi.org/10.1093/biomet/asy008
   Web of Science ®Google Scholar