Don’t Test After Lunch: The Relationship Between Disengagement and the Time of Day That Low-Stakes Testing Occurs

ABSTRACT

When student achievement is assessed, we seek to elicit a student’s maximum performance – a goal requiring the assumption that the student is fully engaged. Otherwise, to the extent that disengagement occurs, test performance is likely to suffer. Effectively managing test-taking disengagement requires an understanding of the testing conditions under which disengagement is more likely. This study investigated the association between disengagement and the time-of-day testing occurred for a widely used low-stakes, computer-adaptive achievement test in grades 2–8. Disengagement was measured using two indicators: response time effort (RTE) and performance decline. Results showed clear evidence for a time-of-day effect for RTE, with the prevalence of students exhibiting low RTE (<.90) tripling during the school day in Math, and nearly doubling in Reading. In contrast, the prevalence of significant performance decline showed little change across time of day. Reasons for differences in results for the two disengagement indicators are discussed.

Disclosure statement

The authors have no known conflicts of interest to disclose.

Notes

¹ Performance decline is challenging to assess because it can be exhibited in multiple forms. Some students may show a gradual decline, while others may show a marked decline that could begin at various points during a test event. In addition, both the limited reliability of individual dichotomously-scored item responses (particularly multiple choice) and heterogeneity of item content render it difficult to detect item-by-item performance change. A practical solution to these constraints is to compare performance on adequately reliable sets of items that have similar content representation. However, the sensitivity of this detection approach depends on the congruence between how the item sets were defined and when performance decline occurred during a test event. Additional research is needed to refine procedures to identifying performance decline.

² The only exception to this pattern was that students in grades 2–5 showed both an increase in the percentage of males and a decrease in initial RIT means during the 12:00 and 1:00 hours. These results suggest the presence of a small selection bias in the afternoon for the younger students.

³ There was a marked increase in the rate of performance decline for Reading at the 1:00 hour in grades 2–5. However, this data point should be interpreted cautiously, as it was the least stable due to its being based on the fewest observations (N = 562) in the graph.