Attending to Variable Interpretations of Assessment Science and Practice

References

• Tavares W, Kuper A, Kulasegaram K, Whitehead C. The compatibility principle: on philosophies in the assessment of clinical competence. Adv Health Sci Educ Theory Pract. 2020;25(4):1003–1018. doi:10.1007/s10459-019-09939-9.
   PubMed Web of Science ®Google Scholar
• Markus KA, Borsboom D. Truth and evidence in validity theory. J Educ Meas. 2013;50(1):110–114. doi:10.2307/23353802.
   Web of Science ®Google Scholar
• Kuper A, Reeves S, Albert M, Hodges BD. Assessment: do we need to broaden our methodological horizons? Med Educ. 2007;41(12):1121–1123. doi:10.1111/j.1365-2923.2007.02945.x.
   PubMed Web of Science ®Google Scholar
• Mislevy RJ. Sociocognitive Foundations of Educational Measurement. New York, NY: Routledge; 2018.
   Google Scholar
• Roy M, Wojcik J, Bartman I, Smee S. Augmenting physician examiner scoring in objective structured clinical examinations: Including the standardized patient perspective. Adv Health Sci Educ Theory Pract. 2021;26(1):313–328. doi:10.1007/s10459-020-09987-6.
   PubMed Web of Science ®Google Scholar
• Ten Cate O, Regehr G. The power of subjectivity in the assessment of medical trainees. Acad Med. 2019;94(3):333–337. doi:10.1097/ACM.0000000000002495.
   PubMed Web of Science ®Google Scholar
• St-Onge C, Young M, Eva KW, Hodges B. Validity: One word with a plurality of meanings. Adv Health Sci Educ Theory Pract. 2017;22(4):853–867. doi:10.1007/s10459-016-9716-3.
   PubMed Web of Science ®Google Scholar
• Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: A practical guide to Kane’s framework. Med Educ. 2015;49(6):560–575. doi:10.1111/medu.12678.
   PubMed Web of Science ®Google Scholar
• Cook DA, Kuper A, Hatala R, Ginsburg S. When assessment data are words: Validity evidence for qualitative educational assessments. Acad Med. 2016;91(10):1359–1369. doi:10.1097/ACM.0000000000001175.
   PubMed Web of Science ®Google Scholar
• Pack R, Lingard L, Watling CJ, Chahine S, Cristancho SM. Some assembly required: Tracing the interpretative work of Clinical Competency Committees. Med Educ. 2019;53(7):723–734. doi:10.1111/medu.13884.
   PubMed Web of Science ®Google Scholar
• Zoanetti N, Pearce J. The potential use of Bayesian Networks to support committee decisions in programmatic assessment. Med Educ. 2021;55(7):808–817. doi:10.1111/medu.14407.
   PubMed Web of Science ®Google Scholar
• Norcini J, Anderson B, Bollela V, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach. 2011;33(3):206–214. doi:10.3109/0142159X.2011.551559.
   PubMed Web of Science ®Google Scholar
• Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med Teach. 2018;40(11):1102–1109. doi:10.1080/0142159X.2018.1500016.
   PubMed Web of Science ®Google Scholar
• Tavares W, Young M, Gauthier G, St-Onge C. The effect of foregrounding intended use on observers’ ratings and comments in the assessment of clinical competence. Acad Med. 2020;95(5):777–785. doi:10.1097/ACM.0000000000003076.
   PubMed Web of Science ®Google Scholar
• Pearce J. In defence of constructivist, utility-driven psychometrics for the ‘post-psychometric era’. Med Educ. 2020;54(2):99–102. doi:10.1111/medu.14039.
   PubMed Web of Science ®Google Scholar
• Tavares W, Hodwitz K, Rowland P, et al. Implicit and inferred: on the philosophical positions informing assessment science. Adv Health Sci Educ Theory Pract. 2021;26(5):1597–1623. doi:10.1007/s10459-021-10063-w.
   PubMed Web of Science ®Google Scholar
• Pearce J. Psychometrics in action, science as practice. Adv Health Sci Educ Theory Pract. 2018;23(3):653–663. doi:10.1007/s10459-017-9789-7.
   PubMed Web of Science ®Google Scholar
• Tavares W, Pearce J, Eva K. Tracing philosophical shifts in health professions assessment. In: Brown MEL, Veen M, Finn GM, eds. Applied Philosophy for Health Professions Education. Gateway East, Singapore: Springer Nature; 2022:67–84. doi:10.1007/978-981-19-1512-3_6.
   Google Scholar
• Boursicot K, Kemp S, Wilkinson T, et al. Performance assessment: consensus statement and recommendations from the 2020 Ottawa Conference. Med Teach. 2021;43(1):58–67. doi:10.1080/0142159X.2020.1830052.
   PubMed Web of Science ®Google Scholar
• Varpio L, MacLeod A. Philosophy of science series: harnessing the multidisciplinary edge effect by exploring paradigms, ontologies, epistemologies, axiologies, and methodologies. Acad Med. 2020;95(5):686–689. doi:10.1097/ACM.0000000000003142.
   PubMed Web of Science ®Google Scholar
• Hodges B. Assessment in the post-psychometric era: learning to love the subjective and collective. Med Teach. 2013;35(7):564–568. doi:10.3109/0142159X.2013.789134.
   PubMed Web of Science ®Google Scholar
• Govaerts MJ, Van der Vleuten CP, Schuwirth LW, Muijtjens AM. Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ Theory Pract. 2007;12(2):239–260. doi:10.1007/s10459-006-9043-1.
   PubMed Web of Science ®Google Scholar
• Van Der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309–317. doi:10.1111/j.1365-2929.2005.02094.x.
   PubMed Web of Science ®Google Scholar
• Gingerich A, Kogan J, Yeates P, Govaerts M, Holmboe E. Seeing the ‘black box’differently: assessor cognition from three research perspectives. Med Educ. 2014;48(11):1055–1068. doi:10.1111/medu.12546.
   PubMed Web of Science ®Google Scholar
• Pearce J, Tavares W. A philosophical history of programmatic assessment: Tracing shifting configurations. Adv Health Sci Educ Theory Pract. 2021;26(4):1291–1310. doi:10.1007/s10459-021-10050-1.
   PubMed Web of Science ®Google Scholar
• Uijtdehaage S, Schuwirth LW. Assuring the quality of programmatic assessment: moving beyond psychometrics. Perspect Med Educ. 2018;7(6):350–351. doi:10.1007/s40037-018-0485-y.
   PubMed Web of Science ®Google Scholar
• Kuhn TS. The Structure of Scientific Revolutions. 3rd ed. Chicago, IL: University of Chicago press; 1996.
   Google Scholar
• Govaerts M, van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47(12):1164–1174. doi:10.1111/medu.12289.
   PubMed Web of Science ®Google Scholar
• Tavares W, Eva KW. Impact of rating demands on rater-based assessments of clinical competence. Educ Prim Care. 2014;25(6):308–318. doi:10.1080/14739879.2014.11730760.
   PubMedGoogle Scholar
• Berendonk C, Stalmeijer RE, Schuwirth LW. Expertise in performance assessment: assessors’ perspectives. Adv Health Sci Educ Theory Pract. 2013;18(4):559–571. doi:10.1007/s10459-012-9392-x.
   PubMed Web of Science ®Google Scholar
• Guyon H, Kop JL, Juhel J, Falissard B. Measurement, ontology, and epistemology: Psychology needs pragmatism-realism. Theory Psychol. 2018;28(2):149–171. doi:10.1177/0959354318761606.
   Web of Science ®Google Scholar
• Schoonenboom J. A performative paradigm for mixed methods research. J Mix Methods Res. 2019;13(3):284–300. doi:10.1177/1558689817722889.
   Web of Science ®Google Scholar
• Johnson RB. Dialectical pluralism: A metaparadigm whose time has come. J Mix Methods Res. 2017;11(2):156–173. doi:10.1177/1558689815607692.
   Web of Science ®Google Scholar
• Kane MT. Validation as a pragmatic, scientific activity. J Educ Meas. 2013;50(1):115–122. doi:10.2307/23353803.
   Web of Science ®Google Scholar
• Tavares W, Rowland P, Dagnone D, McEwen LA, Billett S, Sibbald M. Translating outcome frameworks to assessment programmes: Implications for validity. Med Educ. 2020;54(10):932–942. doi:10.1111/medu.14287.
   PubMed Web of Science ®Google Scholar
• Yvonne Feilzer M. Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. J Mix Methods Res. 2010;4(1):6–16. doi:10.1177/1558689809349691.
   Web of Science ®Google Scholar
• Biesta GJ, Burbules NC. Pragmatism and Educational Research. Lanham, MD: Rowman & Littlefield Publishers; 2003.
   Google Scholar
• Pierce CS, The Pierce Edition Project, ed. The Essential Pierce: Selected Philosophical Writings. Vol. 2 of the Essential Pierce: Selected Philosophical Writings. Bloomington: Indiana University Press; 1992.
   Google Scholar
• James W. Pragmatism: A New Name for Some Old Ways of Thinking. London, UK: Longmans, Green, & Co; 1907:1–132.
   Google Scholar
• James W. Pragmatism and Four Essays from the Meaning of Truth. Cleveland, OH: World Publishing; 1909.
   Google Scholar
• Dewey J, Hickman LA, eds. Pragmatism, education, democracy. In: Alexander TM, ed. The Essential Dewey: Ethics, Logic, Psychology. Vol 1. Bloomington: Indiana University Press; 1999.
   Google Scholar
• Sleeper RW. The Necessity of Pragmatism: John Dewey’s Conception of Philosophy. Champaign: University of Illinois Press; 2001.
   Google Scholar
• Rorty R. Philosophy and the Mirror of Nature. Princeton, NJ: Princeton University Press; 1981.
   Google Scholar
• Rorty R. Consequences of Pragmatism: Essays, 1972–1980. Minneapolis: University of Minnesota Press; 1982.
   Google Scholar
• Veen M, Cianciolo AT. Problems no one looked for: philosophical expeditions into medical education. Teach Learn Med. 2020;32(3):337–344. doi:10.1080/10401334.2020.1748634.
   PubMed Web of Science ®Google Scholar
• Cizek GJ. Validating test score meaning and defending test score use: Different aims, different methods. Assess Educ. 2016;23(2):212–225. doi:10.1080/0969594X.2015.1063479.
   Web of Science ®Google Scholar
• Messick S. The standard problem: Meaning and values in measurement and evaluation. Am Psychol. 1975;30(10):955–966. doi:10.1037/0003-066X.30.10.955.
   Web of Science ®Google Scholar
• Yudkowsky R, Park YS, Downing SM, eds. Assessment in Health Professions Education. New York, NY: Routledge; 2019.
   Google Scholar
• Biesta GJ, van Braak M. Beyond the medical model: thinking differently about medical education and medical education research. Teach Learn Med. 2020;32(4):449–456. doi:10.1080/10401334.2020.1798240.
   PubMed Web of Science ®Google Scholar
• Eva KW, Bordage G, Campbell C, et al. Towards a program of assessment for health professionals: from training into practice. Adv Health Sci Educ Theory Pract. 2016;21(4):897–913. doi:10.1007/s10459-015-9653-6.
   PubMed Web of Science ®Google Scholar
• Varpio L, Paradis E, Uijtdehaage S, Young M. The distinctions between theory, theoretical framework, and conceptual framework. Acad Med. 2020;95(7):989–994. doi:10.1097/ACM.0000000000003075.
   PubMed Web of Science ®Google Scholar