Accounting for Traumatic Historical Events in Educational Randomized Controlled Trials

Abstract

As an example of how historical events may influence the findings and interpretations of a randomized trial, we use a school-based evaluation of a classroom management program that was conducted in a nearby district before and after the shooting of Michael Brown in Ferguson, Missouri (N = 102 teachers and 1,450 students). The findings suggest that the event differentially affected teacher and student response within and across conditions. Black teachers benefited more from the intervention as evidenced by their independently observed classroom management skills and praise-to-reprimand ratios; however, these effects were minimized or disappeared after the event. Additionally, although the intervention equally benefited the academic achievement of Black and White students before the event, the opportunity gap widened after the event. Implications for the design, analysis, and reporting of findings from randomized controlled trials are discussed.

Supplemental data for this article is available online at http://dx.doi.org/10.1080/2372966X.2021.2024768

Impact Statement

This cluster randomized trial conducted before and after the Michael Brown shooting revealed that historical events can differentially effect participants both within and across conditions. Moreover, findings suggest that the Black–White opportunity gap will not be reduced by schools alone.

Keywords:

• history
• opportunity gap
• classroom management
• academic achievement

Associate Editor:

• Tyler Renshaw

DISCLOSURE

The authors have no conflicts of interest to report.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A130143 to the University of Missouri (PI: Keith Herman). The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Notes on contributors

Keith C. Herman

Keith C. Herman is a Curator’s Distinguished Professor in Department of Education, School, & Counseling Psychology at the University of Missouri. He is the Co-Founder and Co-Director of the Missouri Prevention Science Institute. He has an extensive grant and publication record including over 150 peer-reviewed publications in the areas of prevention and early intervention of child emotional and behavior disturbances and culturally-sensitive education interventions.

Nianbo Dong

Nianbo Dong is an Associate Professor and Chair in the Area, Learning, Development, and Psychological Sciences at the University of North Carolina, Chapel Hill. Dr. Dong’s research program centers on developing and applying rigorous quantitative methods to evaluate educational policies, programs, and practice. His current interests in quantitative methodology focus on power analyses of the main, moderation, and mediation effects in multilevel experiments and causal inference. His substantive research focuses on the evaluations of the effectiveness of teacher and principal training programs and early child education programs. His work has been supported by over $5 million of funding as the Principal Investigator or Co-Principal Investigator. He also received the NSF Faculty Early Career award in 2017.

Wendy M. Reinke

Wendy M. Reinke is a Professor in the Department of Education, School, & Counseling Psychology at the University of Missouri. She is the Co-Founder and Co-Director of the Missouri Prevention Science Institute. She has an extensive grant and publication record including over 100 peer-reviewed publications and over $50 million in grant funding in the areas of prevention and early intervention of child emotional and behavior disturbances. She is also the Director of the National Center for Rural School Mental Health and the co-developer and leadership team member for the Family Access Center of Excellence and the Boone County Schools Mental Health Coalition.

Catherine P. Bradshaw

Catherine P. Bradshaw is a Professor and the Associate Dean for Research and Faculty Development at the School of Education and Human Development at the University of Virginia. Prior to her current appointment at U.Va., she was an Associate Professor and the Associate Chair of the Department of Mental Health at the Johns Hopkins Bloomberg School of Public Health, where she maintains an adjunct faculty position. She holds a doctorate in developmental psychology from Cornell University and a master’s of education in counseling and guidance from the University of Georgia.Her primary research interests focus on the development of aggressive behavior and school-based prevention. She collaborates on research projects examining bullying and school climate; the development of aggressive and problem behaviors; effects of exposure to violence, peer victimization, and environmental stress on children; children with emotional and behavioral disorders and autism; and the design, evaluation, and implementation of evidence-based prevention programs in schools. She has led a number of federally funded randomized trials of school-based prevention programs, including Positive Behavioral Interventions and Supports (PBIS) and social–emotional learning curricula. She also has expertise in implementation science and coaching models.