Does classroom technology make a difference? A random assignment study in US classrooms

Abstract

In the 2006–2007 school year, a suburban school district in northwest Arkansas installed state-of-the-art Interwrite educational technology in a randomly selected number of the district's classrooms. This project was designed to explore the benefits of educational technology on student achievement and assess the changes in attitudes and perceptions of participating teachers and students. Although research is emerging about the use of interactive educational technologies at the high school and college levels, the literature is limited regarding the effectiveness of these tools with younger students. In particular, there is little work in the USA in this growing area employing rigorous evaluation methods to assess the effectiveness of technology-based school interventions, particularly of interactive whiteboard programs. Thus, this randomized controlled trial of educational technology supplements this expanding literature base by providing insight into the use of interactive technology in elementary and middle school classrooms in the USA. The results of our analysis suggest that, despite some positive changes in teacher attitude, we should not expect immediate improvements in student achievement from the introduction of this type of technology in the classroom. The experience in this school district suggests that district leaders must take time and be very intentional about overcoming barriers to the adoption of new technology in the classroom.

Keywords:

• interactive technology
• educational technology
• computer-assistive technology
• interactive whiteboard

Notes

Teachers and students were already placed in classrooms within grades and teams prior to the random assignment of the technology sets. Thus, there was not opportunity for the intentional placement of certain teachers or students into the treatment or control groups.

For example, in Pair 1 (which included two district elementary schools), the highest number went to the fourth grade team at Elementary School A. Therefore, automatically, the counterpart third grade team at Elementary School B was selected into the treatment group. The other counterpart teams (i.e. the third grade at Elementary School A and the fourth grade at Elementary School B) in both schools were assigned to the control group.

Each of the fifth and sixth grade teams was assigned a random number, with the teams in each school in each grade receiving the highest number assigned to the treatment group. Then, the team in either fifth or sixth grade with the next highest number was also placed in the treatment group.

The observation instrument is included in the appendix of this paper.

The set of survey items associated with each construct are presented in the appendix.

The set of survey items associated with each construct are presented in the appendix.

From these models we estimated the variance of a student random effect, and a team random effect. The intraclass correlation (ICC) can be considered an estimate of the proportion of variance explained at the team level. For math: ICC = 0.43; Reading: ICC = 0.37; Language: ICC = 0.33; Science: ICC = 0.39; Social Science: ICC = 0.37.

While it may seem trivial to compare technology use in the treatment classrooms to the control classrooms, recall that the control classrooms do still have access to typical classroom technology.

The models were also estimated using grade-specific dummy variables at the team level. These results are not substantively different than those presented here. We also note that we estimated more advanced models that allowed team intercepts or the treatment effect itself to vary by schools in which the teams were located. Estimates of the treatment effects do not substantively differ from those presented here. However, team and school team and school random effects were not estimable in these data, and we were compelled to choose to locate the random effect as a team or school effect. As noted in the text, we present team results here because the randomization occurred at this level.