Which Students Benefit from Computer-Based Individualized Instruction? Experimental Evidence from Public Schools in India

Abstract

This is one of the first studies to evaluate the impact of computer-based individualized instruction in a developing country. We randomly assigned 1,528 students in grades 6–8 in 15 “model” public schools in Rajasthan, India who were using a computer-adaptive learning software to: a control group, in which they were only able to access the activities for their enrolled grade level; or a treatment group, in which they were able to access exercises appropriate for their performance level. After 9 months, computer-based individualized instruction had a null average effect on math achievement. However, treatment students with low initial performance outperformed their control counterparts by 0.22 standard deviations. Our results suggest that computer-based individualized instruction is most beneficial for low-performing students.

Keywords:

• Computer-aided learning
• India
• computer-based individualized instruction
• math instruction

Acknowledgement

We gratefully acknowledge the funding provided by the Douglas B. Marshall, Jr. Family Foundation for this project. We especially thank Karthik Muralidharan for his involvement as a collaborator in the early stages of this project and for subsequent discussions. We thank Pranav Kothari, Aarthi Muralidharan, Sridhar Rajagopalan, Maulik Shah, Nishchal Shukla, and Gayatri Vaidya for making this study possible. We also thank Anuja Venkatachalam for excellent research assistance and field support. This study was registered with the AEA Trial Registry (RCT ID: AEARCTR-0002459). The usual disclaimers apply.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 We initially planned to introduce two more treatment groups. Yet, due to technical difficulties, we abandoned them shortly after baseline and excluded students in those experimental groups from the present study. These interventions are described in our pre-registration plan: https://www.socialscienceregistry.org/trials/2459.

2 The tests can be accessed at: https://bit.ly/3knzgaj (baseline) and https://bit.ly/2E8U0mF (endline).

3 We have no way of knowing why the material for treatment students enrolled in some grades (e.g., grade 6) were concentrated on a specific lower grade (e.g., grade 4), while the material for other grades (e.g., grade 8) were spread across several lower grades (e.g., grades 4–8). It is possible that most students enrolled in lower grades needed reinforcement of basic content, whereas those enrolled in higher grades had more diverse needs.

4 The mapping of exercises to topics was conducted by Educational Initiatives, the developer of the CAL software, prior to the start of the study. The grouping of topics into content domains was conducted by us at the analysis stage.

5 We do find a statistically significant effect of computer-based individualized instruction on items that were first introduced in the endline (supplemental Table A.3, panel A). However, given all other results, we believe that this is likely to have occurred by chance.

6 We report on impacts by within-grade quartile. The intent-to-treat effect for the within-grade bottom half of students is 0.18 SDs (p < 0.01). The effect for the top half of students is −0.05 SDs (p > 0.1).