Don’t Test After Lunch: The Relationship Between Disengagement and the Time of Day That Low-Stakes Testing Occurs

References

• Ackerman, P. L., & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15(2), 163–181. doi:10.1037/a0015719
   PubMed Web of Science ®Google Scholar
• Barry, C. L., & Finney, S. J. (2016). Modeling change in effort across a low-stakes testing session: A latent growth curve modeling approach. Applied Measurement in Education, 29(1), 46–64. doi:10.1080/08957347.2015.1102914
   Web of Science ®Google Scholar
• Borgonovi, F., & Biecek, P. (2016). An international comparison of students’ ability to endure fatigue and maintain motivation during a low-stakes test. Learning and Individual Differences, 49, 128–137. doi:10.1016/j.lindif.2016.06.001
   Web of Science ®Google Scholar
• Cronbach, L. J. (1960). Essentials of psychological testing (2nd ed.). New York, NY: Harper & Row.
   Google Scholar
• DeMars, C. E., Bashkov, B. M., & Socha, A. B. (2013). The role of gender in test-taking motivation under low-stakes conditions. Research & Practice in Assessment, 8, 69–82. http://www.rpajournal.com/dev/wp-content/uploads/2013/11/A4.pdf
   Google Scholar
• Eccles, J. (1983). Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives (pp. 75–146). San Francisco: Freeman.
   Google Scholar
• Flake, J. K., Barron, K. E., Hulleman, C., McCoach, B. D., & Welsh, M. E. (2015). Measuring cost: The forgotten component of expectancy-value theory. Contemporary Educational Psychology, 41, 232–244. doi:10.1016/j.cedpsych.2015.03.002
   Web of Science ®Google Scholar
• Goldhammer, F., Martens, T., Christoph, G., & Lüdtke, O. (2016). Test-taking engagement in PIAAC. OECD Education Working Papers, No. 133. OECD Publishing, Paris.
   Google Scholar
• Guo, H., Rios, J. A., Haberman, S., Liu, O. L., Wang, J., & Paek, I. (2016). A new procedure for detection of students’ rapid guessing responses using response time. Applied Measurement in Education, 29(3), 173–183. doi:10.1080/08957347.2016.1171766
   Web of Science ®Google Scholar
• Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing. Educational Measurement Issues & Practice, 23(1), 17–27. doi:10.1111/j.1745-3992.2004.tb00149.x
   Google Scholar
• Inzlicht, M., & Schmeichel, B. J. (2012). What is ego depletion? Toward a mechanistic revision of the resource model of self-control. Perspectives on Psychological Science, 7(5), 450–463. doi:10.1177/1745691612454134
   Web of Science ®Google Scholar
• Kaur, P., Kumar, H., & Kaushal, S. (2021). Affective state and learning environment based analysis of students’ performance in online assessment. International Journal of Cognitive Computing in Engineering, 2, 12–20. doi:10.1016/j.ijcce.2020.12.003
   Google Scholar
• Knekta, E. (2017). Are all pupils equally motivated to do their best on all tests? Differences in reported test-taking motivation within and between tests with different stakes. Scandinavian Journal of Educational Research, 61(1), 95–111. doi:10.1080/00313831.2015.1119723
   Web of Science ®Google Scholar
• Kroehne, U., Deribo, T., & Goldhammer, F. (2020). Rapid guessing rates across administration mode and test setting. Psychological Test and Assessment Modeling, 62(2), 147–177. doi:10.25656/01:23630
   Google Scholar
• Kuhfeld, M., Soland, J., Register, B., & McEachin, A. (in press). Testing an explanation for summer learning loss: Differential examinee effort between spring and fall. Education Evaluation and Policy Analysis.
   Web of Science ®Google Scholar
• Lindner, M. A., Lüdtke, O., Grund, S., & Köller, O. (2017). The merits of representational pictures in educational assessment: Evidence for cognitive and motivational effects in a time-on-task analysis. Contemporary Educational Psychology, 51, 482–492. doi:10.1016/j.cedpsych.2017.09.009
   Web of Science ®Google Scholar
• Lindner, M. A., Lüdtke, O., & Nagy, G. (2019). The onset of rapid-guessing behavior over the course of testing time: A matter of motivation and cognitive resources. Frontiers in Psychology, 10, 1–14. doi:10.3389/fpsyg.2019.01533
   PubMed Web of Science ®Google Scholar
• Lindner, C., Nagy, G., & Retelsdorf, J. (2018). The need for self-control in achievement tests: Changes in students’ state self-control capacity and effort investment. Social Psychology of Education, 21(5), 1113–1131. doi:10.1007/s11218-018-9455-9
   Web of Science ®Google Scholar
• Lindner, C., & Retelsdorf, J. (2020). Perceived—and not manipulated—self-control depletion predicts students’ achievement outcomes in foreign language assessments. Educational Psychology, 40(4), 490–508. doi:10.1080/01443410.2019.1661975
   Web of Science ®Google Scholar
• Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11. doi:10.2307/1175249
   Google Scholar
• Nagy, G., Ulitzsch, E., & Lindner, M. A. (2022). The role of rapid guessing and test‐taking persistence in modelling test‐taking engagement. Journal of Computer Assisted Learning, 39(3), 751–766. doi:10.1111/jcal.12719
   Web of Science ®Google Scholar
• Pastor, D. A., Ong, T. Q., & Strickman, S. N. (2019). Patterns of solution behavior across items in low-stakes assessments. Educational Assessment, 24(3), 189–212. doi:10.1080/10627197.2019.1615373
   Web of Science ®Google Scholar
• Perkins, B. A., Pastor, D. A., & Finney, S. J. (2021). Between-versus within-examinee variability in test-taking effort and test emotions during a low-stakes test. Applied Measurement in Education, 34(4), 285–300. doi:10.1080/08957347.2021.1987905
   Web of Science ®Google Scholar
• Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40. doi:10.1037/0022-0663.82.1.33
   Web of Science ®Google Scholar
• Pope, N. G. (2016). How the time of day affects productivity: Evidence from school schedules. Review of Economics and Statistics, 98(1), 1–11. doi:10.1162/REST_a_00525
   Web of Science ®Google Scholar
• Rios, J. A., & Liu, O. L. (2017). Online proctored versus unproctored low-stakes internet test administration: Is there differential test-taking behavior and performance? American Journal of Distance Education, 31(4), 226–241. doi:10.1080/08923647.2017.1258628
   Web of Science ®Google Scholar
• Rios, J. A., & Soland, J. (2021). Investigating the impact of noneffortful responses on individual-level scores: Can the effort-moderated IRT model serve as a solution? Applied Psychological Measurement, 45(6), 391–406. doi:10.1177/01466216211013896
   PubMed Web of Science ®Google Scholar
• Schnipke, D. L. (1995). Assessing speededness in computer-based tests using item response times [ Unpublished doctoral dissertation]. Johns Hopkins University.
   Google Scholar
• Schnipke, D. L., & Scrams, D. J. (1997). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34(3), 213–232. doi:10.1111/j.1745-3984.1997.tb00516.x
   Web of Science ®Google Scholar
• Sievertsen, H. H., Gino, F., & Piovesan, M. (2016). Cognitive fatigue influences students’ performance on standardized tests. Proceedings of the National Academy of Sciences, 113(10), 2621–2624. doi:10.1073/pnas.1516947113
   PubMed Web of Science ®Google Scholar
• Sjosten-Bell, W. (2005). Influence of time-of-day on student performance on mathematical algorithms. [ Unpublished master’s thesis]. Dominican University of California.
   Google Scholar
• Weiss, D. J., & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361–375. doi:10.1111/j.1745-3984.1984.tb01040.x
   Web of Science ®Google Scholar
• Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. doi:10.1006/ceps.1999.1015
   PubMed Web of Science ®Google Scholar
• Wise, S. L. (2015). Effort analysis: Individual score validation of achievement test data. Applied Measurement in Education, 28(3), 237–252. doi:10.1080/08957347.2015.1042155
   Web of Science ®Google Scholar
• Wise, S. L. (2017). Rapid-guessing behavior: Its identification, interpretations, and implications. Educational Measurement Issues & Practice, 36(4), 52–61. doi:10.1111/emip.12165
   Google Scholar
• Wise, S. L., & Kingsbury, G. G. (2016). Modeling student test-taking motivation in the context of an adaptive achievement test. Journal of Educational Measurement, 53(1), 86–105. doi:10.1111/jedm.12102
   Web of Science ®Google Scholar
• Wise, S. L., & Kingsbury, G. G. (2022). Performance decline as an indicator of generalized test-taking disengagement. Applied Measurement in Education, 35(4), 272–286. doi:10.1080/08957347.2022.2155651
   Web of Science ®Google Scholar
• Wise, S. L., Kingsbury, G. G., & Langi, M. L. (2023). Change in engagement during test events: An argument for weighted scoring? Applied Measurement in Education, 36(4), 340–354.
   PubMed Web of Science ®Google Scholar
• Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. doi:10.1207/s15324818ame1802_2
   Web of Science ®Google Scholar
• Wise, S. L., & Kuhfeld, M. R. (2020). A cessation of measurement: Identifying test taker disengagement using response time. In M. J. Margolis & R. A. Feinberg (Eds.), Integrating timing considerations to improve testing practices (pp. 150–164). New York: Routledge. doi:10.4324/9781351064781
   Google Scholar
• Wise, S. L., & Kuhfeld, M. R. (2021a). A method for identifying partial test-taking engagement. Applied Measurement in Education, 34(2), 150–161. doi:10.1080/08957347.2021.1890745
   Web of Science ®Google Scholar
• Wise, S. L., & Kuhfeld, M. R. (2021b). Using retest data to evaluate and improve effort-moderated scoring. Journal of Educational Measurement, 58(1), 130–149. doi:10.1111/jedm.12275
   Web of Science ®Google Scholar
• Wise, S. L., & Ma, L. (2012, April). Setting response time thresholds for a CAT item pool: The normative threshold method. [ Paper presentation]. National Council on Measurement in Education 2012 Annual Meeting, Vancouver, Canada.
   Google Scholar
• Wise, S. L., Ma, L., Kingsbury, G. G., & Hauser, C. (2010, May). An investigation of the relationship between time of testing and test-taking effort. [ Paper presentation]. National Council on Measurement in Education 2010 Annual Meeting, Denver, Colorado.
   Google Scholar
• Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: Implications for test development and measurement practice. Applied Measurement in Education, 22(2), 185–205. doi:10.1080/08957340902754650
   Web of Science ®Google Scholar
• Wise, S. L., & Smith, L. F. (2011). A model of examinee test-taking effort. In J. A. Bovaird, K. F. Geisinger, & C. W. Buckendal (Eds.), High-stakes testing in education: Science and practice in K-12 settings (pp. 139–153). American Psychological Association. doi:10.1037/12330-009
   Google Scholar
• Wise, S. L., Soland, J., & Dupray, L. M. (2021). The impact of technology-enhanced items on test-taking disengagement. Journal of Applied Testing Technology, 22(1), 12–24. Retrieved from http://www.jattjournal.com/index.php/atp/article/view/155941
   Google Scholar
• Wolf, L. F., & Smith, J. K. (1995). The consequence of consequence: Motivation, anxiety, and test performance. Applied Measurement in Education, 8(3), 227–242. doi:10.1207/s15324818ame0803_3
   Web of Science ®Google Scholar
• Wolf, L. F., Smith, J. K., & Birnbaum, M. E. (1995). Consequence of performance, test motivation, and mentally taxing items. Applied Measurement in Education, 8(4), 341–351. doi:10.1207/s15324818ame0804_4
   Web of Science ®Google Scholar
• Zagar, R., & Bowers, N. D. (1983). The effect of time of day on problem solving and classroom behavior. Psychology in the Schools, 20(3), 337–345. doi:10.1002/1520-6807(198307)20:3<337:AID-PITS2310200314>3.0.CO;2-C
   Web of Science ®Google Scholar