Explaining performance decline over the course of taking comprehensive proficiency tests: the roles of effort and omission propensity

Abstract

In order to ensure content validity by covering a broad range of content domains, the testing times of some educational large-scale assessments last up to a total of two hours or more. Performance decline over the course of taking the test has been extensively documented in the literature. It can occur due to increases in the numbers of (a) incorrect responses or (b) missing responses. Both reasons can be regarded as the result of decreasing effort. We investigated the roles of self-reported effort and omission propensity for performance decline over the course of taking a comprehensive educational large-scale assessment in Germany. Latent change score models showed that the effects of initial effort and change in effort on decline in performance remained almost unchanged when we added omission propensity and change in omission propensity as predictors of performance decline. This indicates that self-reported effort and omission propensity capture different aspects of performance decline. This finding held in both proficiency domains we investigated: math and science. When developing strategies for mitigating threats to test score validity, the two constructs (i.e., effort and omission propensity) may be considered separately.

Keywords:

• Test-taking effort
• student proficiency
• performance decline
• nonresponse
• omission propensity
• educational large-scale assessment
• latent change score model