2,214
Views
6
CrossRef citations to date
0
Altmetric
Articles

In defence of psychometric measurement: a systematic review of contemporary self-report feedback inventories

ORCID Icon & ORCID Icon

References

*=inventory retained for review in Table 1

  • *Adams, A.-M., Wilson, H., Money, J., Palmer-Conn, S., & Fearn, J. (2020). Student engagement with feedback and attainment: the role of academic self-efficacy. Assessment & Evaluation in Higher Education, 45(2), 317–329. https://doi.org/10.1080/02602938.2019.1640184
  • Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T
  • Ajzen, I. (2005). Attitudes, personality and behavior. (2nd ed.). Open University Press.
  • Allport, G. W. (1935). Attitudes. In C. Murchison (Ed.), A Handbook of Social Psychology (pp. 798–844). Clark University Press.
  • Ashford, S. J., Stobbeleir, K. D., & Nujella, M. (2016). To seek or not to seek: Is that the only question? Recent developments in feedback-seeking literature. Annual Review of Organizational Psychology and Organizational Behavior, 3(1), 213–239. https://doi.org/10.1146/annurev-orgpsych-041015-062314
  • Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
  • Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54(2), 199–231. https://doi.org/10.1111/j.1464-0597.2005.00205.x
  • Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press.
  • Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061
  • Brown, G. T. L., Harris, L. R., O'Quin, C., & Lane, K. E. (2017). Using multi-group confirmatory factor analysis to evaluate cross-cultural research: Identifying and understanding non-invariance. International Journal of Research & Method in Education, 40(1), 66–90. https://doi.org/10.1080/1743727X.2015.1070823
  • Brown, G. T. L., Peterson, E. R., & Irving, S. E. (2009). Beliefs that make a difference: Adaptive and maladaptive self-regulation in students’ conceptions of assessment. In D. M. McInerney, G. T. L. Brown, & G. A. D. Liem (Eds.), Student perspectives on assessment: What students can tell us about assessment for learning (pp. 159–186). Information Age Publishing.
  • *Brown, G. T. L., Peterson, E. R., & Yao, E. S. (2016). Student conceptions of feedback: Impact on self-regulation, self-efficacy, and academic achievement. The British Journal of Educational Psychology, 86(4), 606–629. https://doi.org/10.1111/bjep.12126
  • Brown, G. T. L., & Shulruf, B. (2023). Response option design in surveys. In L. R. Ford & T. A. Scandura (Eds.), The SAGE Handbook of Survey Development and Application. Sage.
  • Bryant, F. B., & Yarnold, P. R. (1995). Principal-components analysis and exploratory and confirmatory factor analysis. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding multivariate statistics (pp. 99–136). APA.
  • Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245
  • Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures – the issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. https://doi.org/10.1037/0033-2909.105.3.456
  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multi-trait multi-method matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
  • Carpenter, S. K., Witherby, A. E., & Tauber, S. K. (2020). On students’ (Mis)judgments of learning and teaching effectiveness. Journal of Applied Research in Memory and Cognition, 9(2), 137–151. https://doi.org/10.1016/j.jarmac.2019.12.009
  • Cizek, G. J. (2020). Validity: An integrated approach to test score meaning and use. Routledge.
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. (2nd ed.). Routledge.
  • Crafts, L. W., & Gilbert, R. W. (1935). The effect of knowledge of results on maze learning and retention. Journal of Educational Psychology, 26(3), 177–187. https://doi.org/10.1037/h0061908
  • Cronbach, L. J. (1988). Five perspectives on the validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Lawrence Erlbaum.
  • Cudeck, R., & MacCallum, R. C. (Eds.) (2007). Factor analysis at 100: Historical developments and future directions. LEA.
  • De Stobbeleir, K. E. M., Ashford, S. J., & Buyens, D. (2011). Self-regulation of creativity at work: The role of feedback-seeking behavior in creative performance. Academy of Management Journal, 54(4), 811–831. https://doi.org/10.5465/amj.2011.64870144
  • DiStefano, C., Zhu, M., & Mîndrilă, D. (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1–11. https://doi.org/10.7275/da8t-4g52
  • Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5(3), 69–106. https://doi.org/10.1111/j.1529-1006.2004.00018.x
  • Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. LEA.
  • Esarey, J., & Valdes, N. (2020). Unbiased, reliable, and valid student evaluations can still be unfair. Assessment & Evaluation in Higher Education, 45(8), 1106–1120. https://doi.org/10.1080/02602938.2020.1724875
  • Fan, X., Miller, B. C., Park, K.-E., Winward, B. W., Christensen, M., Grotevant, H. D., & Tai, R. H. (2006). An exploratory study about inaccuracy and invalidity in adolescent self-report surveys. Field Methods, 18(3), 223–244. https://doi.org/10.1177/152822X06289161
  • Fatima, S., Ali, M., & Saad, M. I. (2022). The effect of students’ conceptions of feedback on academic self-efficacy and self-regulation: evidence from higher education in Pakistan. Journal of Applied Research in Higher Education, 14(1), 180–199. https://doi.org/10.1108/JARHE-07-2020-0209
  • Fong, C. J., & Schallert, D. L. (2023). “Feedback to the future”: Advancing motivational and emotional perspectives in feedback research. Educational Psychologist, 1–16. https://doi.org/10.1080/00461520.2022.2134135
  • Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
  • Gibbs, G., & Simpson, C. (2003). Measuring the response of students to assessment: The assessment experience questionnaire. Paper Presented at the 11th Improving Student Learning Symposium.
  • Guo, W. (2020). Grade-level differences in teacher feedback and students’ self-regulated learning. Frontiers in Psychology, 11, 783. https://doi.org/10.3389/fpsyg.2020.00783
  • Harris, L. R., & Brown, G. T. L. (2010). Mixing interview and questionnaire methods: Practical problems in aligning data. Practical Assessment, Research and Evaluation, 15(1), 2. https://doi.org/10.7275/959j-ky83
  • Harris, L. R., & Brown, G. T. L. (2018). Understanding and supporting student self-assessment. Routledge.
  • Harris, L. R., Brown, G. T. L., & Dargusch, J. (2018). Not playing the game: student assessment resistance as a form of agency. The Australian Educational Researcher, 45(1), 125–140. https://doi.org/10.1007/s13384-018-0264-0
  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
  • Hattie, J., Hodis, F. A., & Kang, S. H. K. (2020). Theories of motivation: Integration and ways forward. Contemporary Educational Psychology, 61, 101865. https://doi.org/10.1016/j.cedpsych.2020.101865
  • Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? The Behavioral and Brain Sciences, 33(2–3), 61–135. https://doi.org/10.1017/S0140525X0999152X
  • Hernández Rivero, V. M., Bonilla, P. J. S., & Alonso, J. J. S. (2021). Feedback and self-regulated learning in higher education. Revista de Investigación Educativa, 39(1), 227–248. https://doi.org/10.6018/rie.423341
  • *Howell, J. L., & Shepperd, J. A. (2016). Establishing an information avoidance scale. Psychological Assessment, 28(12), 1695–1708. https://doi.org/10.1037/pas0000315
  • International Test Commission. (2017). The ITC guidelines for translating and adapting tests. https://tinyurl.com/t6z2v42n
  • Irving, S. E., Peterson, E. R., & Brown, G. T. L. (2007). Student Conceptions of Feedback: A study of New Zealand secondary students. Paper Presented to the Biennial Conference of the European Association for Research in Learning and Instruction, Budapest, Hungary.
  • *Jellicoe, M., & Forsythe, A. (2019). The development and validation of the feedback in learning scale (FLS). Frontiers in Education, 4, 84. https://doi.org/10.3389/feduc.2019.00084
  • Jones, L. V., & Thissen, D. (2006). A history and overview of psychometrics. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26, pp. 1–27). Elsevier.
  • Jöreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70(351a), 631–639. https://doi.org/10.1080/01621459.1975.10482485
  • Jöreskog, K. G., & van Thillo, M. (1972). LISREL: A general computer program for estimating a linear structural equation system involving multiple indicators of unmeasured variables (RB-72-56). Educational Testing Service.
  • Joughin, G., Boud, D., Dawson, P., & Tai, J. (2021). What can higher education learn from feedback seeking behaviour in organisations? Implications for feedback literacy. Assessment & Evaluation in Higher Education, 46(1), 80–91. https://doi.org/10.1080/02602938.2020.1733491
  • Karabenick, S. A., & Berger, J.-L. (2013). Help seeking as a self-regulated learning strategy. In Applications of self-regulated learning across diverse disciplines: A tribute to Barry J. Zimmerman (pp. 237–261). Information Age Publishing.
  • Karabenick, S. A., Woolley, M. E., Friedel, J. M., Ammon, B. V., Blazevski, J., Bonney, C. R., Groot, E. D., Gilbert, M. C., Musu, L., Kempler, T. M., & Kelly, K. L. (2007). Cognitive processing of self-report items in educational research: Do they think what we mean? Educational Psychologist, 42(3), 139–151. https://doi.org/10.1080/00461520701416231
  • Kim, S.-I., Lee, M.-J., Chung, Y., & Bong, M. (2010). Comparison of brain activation during norm-referenced versus criterion-referenced feedback: The role of perceived competence and performance-approach goals. Contemporary Educational Psychology, 35(2), 141–152. https://doi.org/10.1016/j.cedpsych.2010.04.002
  • *King, P. E., Schrodt, P., & Weisel, J. J. (2009). The instructional feedback orientation scale: Conceptualizing and validating a new measure for assessing perceptions of instructional feedback. Communication Education, 58(2), 235–261. https://doi.org/10.1080/03634520802515705
  • Kirschner, P. A., & Hendrick, C. (2020). Why independent learning is not a good way to become an independent learner. In P. A. Kirschner & C. Hendrick (Eds.), How learning happens: Seminal works in educational psychology and what they mean in practice (pp. 66–73). Routledge.
  • *König, N., & Puca, R. M. (2019). The German feedback socialization inventory. European Journal of Psychological Assessment, 35(4), 544–554. https://doi.org/10.1027/1015-5759/a000409
  • Krosnick, J. A., & Presser, S. (2009). Question and questionnaire design. Elsevier.
  • *Kyaruzi, F., Strijbos, J. W., Ufer, S., & Brown, G. T. L. (2019). Students’ formative assessment perceptions, feedback use and mathematics performance in secondary schools in Tanzania. Assessment in Education: Principles, Policy and Practice, 26(3), 278–302. https://doi.org/10.1080/0969594X.2019.1593103
  • Leckey, J., & Neill, N. (2001). Quantifying quality: The importance of student feedback. Quality in Higher Education, 7(1), 19–32. https://doi.org/10.1080/13538320120045058
  • *Linderbaum, B. A., & Levy, P. E. (2010). The development and validation of the feedback orientation scale (FOS). Journal of Management, 36(6), 1372–1405. https://doi.org/10.1177/0149206310373145
  • Lindsay, D. S. (2015). Replication in psychological science. Psychological Science, 26(12), 1827–1832. https://doi.org/10.1177/0956797615616374
  • Lipnevich, A. A., & Panadero, E. (2021). A review of feedback models and theories: Descriptions, definitions, and conclusions. Frontiers in Education, 6, 100416. https://doi.org/10.3389/feduc.2021.720195
  • Lipnevich, A. A., Berg, D. A. G., & Smith, J. K. (2016). Toward a model of student response to feedback. In G. T. L. Brown & L. R. Harris (Eds.), The handbook of human and social conditions in assessment (pp. 169–185). Routledge.
  • Little, T., Dawson, P., Boud, D., & Tai, J. (2023). Can students’ feedback literacy be improved? A scoping review of interventions. Assessment & Evaluation in Higher Education, 1–14. https://doi.org/10.1080/02602938.2023.2177613
  • Lundgren, E., & Eklöf, H. (2020). Within-item response processes as indicators of test-taking effort and motivation. Educational Research and Evaluation, 26(5–6), 275–301. https://doi.org/10.1080/13803611.2021.1963940
  • Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research. Perspectives on Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688
  • Marsh, H. W., Hau, K.-T., Artelt, C., Baumert, J., & Peschar, J. L. (2006). OECD's brief self-report measure of educational psychology’s most useful affective constructs: Cross-cultural, psychometric comparisons across 25 countries. International Journal of Testing, 6(4), 311–360. https://doi.org/10.1207/s15327574ijt0604_1
  • McCoach, D. B., Gable, R. K., & Madura, J. P. (2013). Instrument development in the affective domain: school and corporate applications. Springer.
  • McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
  • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). MacMillan.
  • Meyer, G. M., Marco-Pallarés, J., Boulinguez, P., & Sescousse, G. (2021). Electrophysiological underpinnings of reward processing: Are we exploiting the full potential of EEG? NeuroImage, 242, 118478. https://doi.org/10.1016/j.neuroimage.2021.118478
  • Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology. Contemporary Educational Psychology, 25(1), 3–53. https://doi.org/10.1006/ceps.1999.1019
  • Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
  • Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., Halpern, D. E., Loehlin, J. C., Perloff, R., Sternberg, R. J., & Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51(2), 77–101. https://doi.org/10.1037/0003-066X.51.2.77
  • Nieminen, J. H., & Carless, D. (2022). Feedback literacy: a critical review of an emerging concept. Higher Education, https://doi.org/10.1007/s10734-022-00895-9
  • Panadero, E., & Lipnevich, A. A. (2022). A review of feedback models and typologies: Towards an integrative model of feedback elements. Educational Research Review, 35, 100416. https://doi.org/10.1016/j.edurev.2021.100416
  • Pat-El, R. J., Tillema, H., Segers, M., & Vedder, P. (2013). Validation of Assessment for Learning Questionnaires for teachers and students. The British Journal of Educational Psychology, 83(Pt 1), 98–113. https://doi.org/10.1111/j.2044-8279.2011.02057.x
  • Philips, D. C., & Burbules, N. C. (2000). Postpositivism and educational research. Rowman & Littlefield Publishers.
  • Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28(1), 4–13. https://doi.org/10.1002/bs.3830280103
  • Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387–415. https://doi.org/10.1080/02602930500099193
  • *Rui, Y., & Muthukrishnan, P. (2019). Growth mindset and students’ perception of their English Language teachers’ feedback as predictors of language proficiency of the EFL learners. Asian EFL Journal, 23(3), 32–60. https://www.asian-efl-journal.com/main-editions-new/2019-main-journal/volume-23-issue-3-2-2019/index.htm
  • Schacter, D. L. (1999). The seven sins of memory: Insights from psychology and cognitive neuroscience. The American Psychologist, 54(3), 182–203. https://doi.org/10.1037/0003-066X.54.3.182
  • Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453. https://doi.org/10.3102/00346543075003417
  • Stahlschmidt, S., & Stephen, D. (2020). Comparison of Web of Science, Scopus and Dimensions databases (KB Forschungspoolprojekt 2020). https://bibliometrie.info/downloads/DZHW-Comparison-DIM-SCP-WOS.PDF
  • Steelman, L. A., Levy, P. E., & Snell, A. F. (2004). The feedback environment scale: Construct definition, measurement, and validation. Educational and Psychological Measurement, 64(1), 165–184. https://doi.org/10.1177/0013164403258440
  • Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4(1), 25–29. https://doi.org/10.1037/h0071663
  • Van der Kleij, F. M., & Lipnevich, A. A. (2021). Student perceptions of assessment feedback: a critical scoping review and call for research. Educational Assessment, Evaluation and Accountability, 33(2), 345–373. https://doi.org/10.1007/s11092-020-09331-x
  • van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical guide to modelling cognitive processes. Academic Press.
  • Vogl, E., & Pekrun, R. (2016). Emotions that matter to achievement: Student feelings about assessment. In G. T. L. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 111–128). Routledge.
  • Williams, J., & Kane, D. (2008). Exploring the national student survey: Assessment and feedback issues. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=c38acc136c17c130a64845c31cef1d2c42385491
  • Wise, S. L., & Cotten, M. R. (2009). Test-taking effort and score validity: The influence of student conceptions of assessment. In D. M. McInerney, G. T. L. Brown, & G. A. D. Liem (Eds.), Student perspectives on assessment: What students can tell us about assessment for learning (pp. 187–205). Information Age Publishing.
  • Wisniewski, B., Zierer, K., & Hattie, J. (2019). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. https://doi.org/10.3389/fpsyg.2019.03087
  • Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation, 12(3), 1–26. https://doi.org/10.7275/mhqa-cd89
  • Yan, Z., & Brown, G. T. L. (2017). A cyclical self-assessment process: Towards a model of how students engage in self-assessment. Assessment & Evaluation in Higher Education, 42(8), 1247–1262. https://doi.org/10.1080/02602938.2016.1260091
  • Yan, Z., Brown, G. T. L., Lee, J. C.-K., & Qiu, X.-L. (2020). Student self-assessment: Why do they do it? Educational Psychology, 40(4), 509–532. https://doi.org/10.1080/01443410.2019.1672038
  • Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., Paunesku, D., Romero, C., Flint, K., Roberts, A., Trott, J., Iachan, R., Buontempo, J., Yang, S. M., Carvalho, C. M., … Dweck, C. S. (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573(7774), 364–369. https://doi.org/10.1038/s41586-019-1466-y
  • *Zhu, J., & Mok, M. M. C. (2018). Predicting primary students’ self-regulated learning by their prior achievement, interest, personal best goal orientation and teacher feedback. Educational Psychology, 38(9), 1106–1128. https://doi.org/10.1080/01443410.2018.1497775
  • Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2
  • Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. Lissitz (Ed.), The concept of validity (pp. 65–82). Information Age Publishers.