Evaluating the Impact of Supplemental Computer-Assisted Math Instruction in Elementary School: A Conceptual Replication

Abstract

Despite the surge in efficacy studies, the relative effectiveness of any two math interventions is largely unknown and replications are rare. To ensure that students benefit from high-quality learning experiences, the current conceptual replication of Wang and Woodworth, a randomized control trial, evaluated the relative impacts of two computer programs used in a school district as supplements to students’ regular education math instruction, DreamBox Learning and Zearn Math. For 115 students in kindergarten and first grade, data were collected before and after an 11-week implementation period. Measures of effect size, but not null hypothesis statistical tests, suggested that participants in the DreamBox condition had higher test scores on posttest numeracy and geometry (Hedges g = .32 and .14). There was also a statistically significant group by pretest interaction, evidence of “Matthew Effects” for numeracy. Although some support for adopting DreamBox as a supplemental method for improving math competencies of students in early elementary school was provided, more research is needed as the true effect may be smaller than what the present study reports. The study’s implications are discussed.

Keywords:

• Computer-assisted instruction
• numeracy
• geometry
• efficacy study
• replication

Acknowledgments

The author wishes to acknowledge the invaluable support and contributions of the research department, administrators, teachers, and students of the Pasco County School District in Florida. Without their support and that of my research team led by Dr. Anna Garcia, this project would not have been possible. I would also like to thank Dr. Sara Smith and the peer reviewers for their constructive feedback. Finally, I would like to thank my institution whose support made this study possible. Thank you!

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 Percents do not equal 100% due to rounding.

2 English learners are defined as students who meet the Title III criteria of the Every Student Succeeds Act of 2015 to receive supplemental services in the public school system to improve their English language proficiency and academic achievement.