Technology enhanced assessment: Ottawa consensus statement and recommendations

References

• Ahsan K, Akbar S, Kam B. 2021. Contract cheating in higher education: a systematic literature review and future research agenda. Assess Eval High Educ. 47:523–539.
   Web of Science ®Google Scholar
• Alexander B, Ashford-Rowe K, Barajas-Murphy N, Dobbin G, Knott J, McCormack M, Pomerantz J, Seilhamer R, Weber N. 2019. EDUCAUSE horizon report: 2019 higher education edition. Louisville: EDUCAUSE; [accessed 2022 Jan 21]. https://library.educause.edu/-/media/files/library/2019/4/2019horizonreport.pdf.
   Google Scholar
• Amin Z, Boulet JR, Cook DA, Ellaway R, Fahal A, Kneebone R, Maley M, Ostergaard D, Ponnamperuma G, Wearn A, et al. 2011. Technology-enabled assessment of health professions education: Consensus statement and recommendations from the Ottawa 2010 conference. Med Teach. 33(5):364–369.
   PubMedGoogle Scholar
• Andreou V, Peters S, Eggermont J, Wens J, Schoenmakers B. 2021. Remote versus on-site proctored exam: comparing student results in a cross sectional-study. BMC Med Educ. 21:624.
   PubMed Web of Science ®Google Scholar
• Archer E, Prinsloo P. 2020. Speaking the unspoken in learning analytics: troubling the defaults. Assess Eval Higher Educ. 45(6):888–900.
   Web of Science ®Google Scholar
• Aristovnik A, Keržič D, Ravšelj D, Tomaževič N, Umek L. 2020. Impacts of the COVID-19 pandemic of life of higher education students: a global perspective. Sustainability. 12(20):8438.
   Web of Science ®Google Scholar
• Auxier B, Anderson M, Kumar M. 2019. 10 tech-related trends that shaped the decade. Pew Research; [accessed 2022 Jan 21]. https://www.pewresearch.org/fact-tank/2019/12/20/10-tech-related-trends-that-shaped-the-decade/.
   Google Scholar
• Awdry R, Dawson P, Sutherland-Smith W. 2021. Contract cheating: to legislate or not to legislate – is that the question? Assess Eval High Educ. DOI:10.1080/02602938.2021.1957773
   Web of Science ®Google Scholar
• Bali M. 2021. On proctoring & internalised oppression #Against Surveillance; [accessed 2022 Jan 21]. https://blog.mahabali.me/educational-technology-2/on-proctoring-internalized-oppression-againstsurveillance/.
   Google Scholar
• Bengtsson L. 2019. Take-home exams in higher education: a systematic review. Educ Sci. 9(4):267.
   Web of Science ®Google Scholar
• Bennett S, Maton K, Kervin L. 2008. The ‘digital natives’ debate: a critical review of the evidence. BJET. 39(5):775–786.
   Web of Science ®Google Scholar
• Biggs J, Tang C. 2011. Teaching for quality learning at university. Maidenhead: Open University Press.
   Google Scholar
• Boud D. 2000. Sustainable assessment: rethinking assessment for the learning society. Stud Contin Educ. 22(2):151–167.
   Google Scholar
• Boud D, Molloy E. 2013. Rethinking models of feedback for learning: the challenge of design. Assess Eval High Educ. 38(6):698–712.
   Web of Science ®Google Scholar
• Boud D, Soler R. 2016. Sustainable assessment revisited. Assess Eval High Educ. 41(3):400–413.
   Web of Science ®Google Scholar
• Boursicot K, Kemp S, Ong TH, Wijaya L, Goh SH, Freeman K, Curran I. 2020. Conducting a high-stakes OSCE in a COVID-19 environment. MedEdPublish. 9(1):54.
   PubMedGoogle Scholar
• Boursicot K, Kemp S, Wilkinson T, Findyartini A, Canning C, Cilliers F, Fuller R. 2021. Performance assessment: consensus statement and recommendations from the 2020 Ottawa conference. Med Teach. 43(1):58–67.
   PubMed Web of Science ®Google Scholar
• Broos T, Pinxten M, Delporte M, Verbert K, De Laet T. 2020. Learning dashboards at scale: early warning and overall first year experience. Assess Eval High Educ. 45(6):855–874.
   Web of Science ®Google Scholar
• Broussard M. 2018. Artificial unintelligence. How computers misunderstand the world. Cambridge (MA): The MIT Press.
   Google Scholar
• Clarke JL, Boud D. 2018. Refocusing portfolio assessment: curating for feedback and portrayal. Innov Educ Teach Int. 55(4):479–486.
   Web of Science ®Google Scholar
• Cleland J, McKimm J, Fuller R, Taylor D, Janczukowicz J, Gibbs T. 2020. Adapting to the impact of COVID-19: sharing stories, sharing practice. Med Teach. 42(7):772–775.
   PubMed Web of Science ®Google Scholar
• Collares CF, Cecilio-Fernandes D. 2019. When I say … computerised adaptive testing. Med Educ53(2):115–116.
   PubMed Web of Science ®Google Scholar
• Cook DA, Ellaway RH. 2015. Evaluating technology-enhanced learning: a comprehensive framework. Med Teach. 37(10):961–970.
   PubMed Web of Science ®Google Scholar
• Cumbo B, Selwyn N. 2022. Using participatory design approaches in educational research. Int J Res Method Educ. 45(1):60–72.
   Web of Science ®Google Scholar
• Damgaard MT, Nielsen HS. 2018. Nudging in education. Econ Educ Rev. 64:313–342.
   Web of Science ®Google Scholar
• Daniels VJ, Strand AC, Lai H, Hillier T. 2019. Impact of tablet-scoring and immediate score sheet review on validity and educational impact in an internal medicine residency objective structured clinical exam (OSCE). Med Teach. 41(9):1039–1044.
   PubMed Web of Science ®Google Scholar
• Darnton R, Lopez T, Anil M, Ferdinand J, Jenkins M. 2021. Medical students consulting from home: a qualitative evaluation of a tool for maintaining student exposure to patients during lockdown. Med Teach. 43(2):160–167.
   PubMed Web of Science ®Google Scholar
• Davies N, Walker T, Joynes V. 2010. Assessment and learning in practice settings (ALPS) – implementing a large scale mobile learning programme. A report; [accessed 2022 Jan 21]. http://www.alps-cetl.ac.uk/documents/ALPS%20IT%20Report.pdf.
   Google Scholar
• Divami. 2021. Top 5 principles of EdTech design; [accessed 2022 Jan 21]. https://www.divami.com/blog/top-5-principles-of-edtech-design/.
   Google Scholar
• Driessen E. 2017. Do portfolios have a future? Adv Health Sci Educ Theory Pract. 22(1):221–228.
   PubMed Web of Science ®Google Scholar
• Durning SJ, Dong T, Ratcliffe T, Schuwirth L, Artino AR, Boulet JR, Eva K. 2016. Comparing open-book and closed-book examinations: a systematic review. Acad Med. 91(4):583–599.
   PubMed Web of Science ®Google Scholar
• EDUCAUSE. 2021. Top IT issues, 2021: emerging from the pandemic; [accessed 2022 Jan 21]. https://er.educause.edu/articles/2020/11/top-it-issues-2021-emerging-from-the-pandemic.
   Google Scholar
• Ellaway R, Cleland J, Tolsgaard M. 2020. What we learn in time of pestilence. Adv Health Sci Educ Theory Pract. 25(2):259–261.
   PubMed Web of Science ®Google Scholar
• Er H, Nadarajah V, Wong P, Mitra N, Ibrahim Z. 2020. Practical considerations for online open book examinations in remote settings. MedEdPublish. 9(1):153.
   PubMedGoogle Scholar
• Er HM, Nadarajah VD, Hays RB, Yusoff N, Loh KLY. 2019. Lessons learnt from the development and implementation of an online assessment system for medical science programmes. Med Sci Educ. 29(4):1103–1108.
   PubMedGoogle Scholar
• Farrokhi F, Mohebbi SZ, Farrokhi F, Khami MR. 2021. Impact of COVID-19 on dental education–a scoping review. BMC Med Educ. 21(1):1–12.
   PubMed Web of Science ®Google Scholar
• Folger D, Merenmies J, Sjöberg L, Pyörälä E. 2021. Hurdles for adopting mobile learning devices at the outset of clinical courses. BMC Med Educ. 21(1):594.
   PubMed Web of Science ®Google Scholar
• Foster C, Francis P. 2020. A systematic review on the deployment and effectiveness of data analytics in higher education to improve student outcomes. Assessment and Evaluation in higher education to improve student outcomes. Assess Eval High Educ. 45(6):822–841.
   Web of Science ®Google Scholar
• Foster E, Siddle R. 2020. The effectiveness of learning analytics for identifying at-risk students in higher education. Assess Eval Higher Educ. 45:842–854.
   Google Scholar
• Fuller R, Joynes V. 2015. Enhancement or replacement? Understanding how legitimised use of mobile learning resources is shaping how healthcare students are learning. J EAHIL. 11(2):7–10.
   Google Scholar
• Fuller R, Joynes V, Cooper J, Boursicot K, Roberts T. 2020. Could Covid-19 be our ‘there is no alternative’ (TINA) opportunity to enhance assessment? Med Teach. 42(7):781–786.
   PubMed Web of Science ®Google Scholar
• Gartner. 2015. Technology and business in 2030; [accessed 2022 Jan 21]. https://www.gartner.com/smarterwithgartner/technology-and-business-in-2030.
   Google Scholar
• Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, Thammasitboon S, Park S, Pawlikowska T, Rees EL, et al. 2020. Developments in medical education in response to the COVID-19 pandemic: a rapid BEME systematic review: BEME Guide No. 63. Med Teach. 42(11):1202–1215.
   PubMed Web of Science ®Google Scholar
• Gormley GJ, Johnston J, Thomson C, McGlade K. 2012. Awarding global grades in OSCEs: evaluation of a novel eLearning resource for OSCE examiners. Med Teach. 34(7):587–589.
   PubMed Web of Science ®Google Scholar
• Grafton-Clarke C, Uraiby H, Gordon M, Clarke N, Rees E, Park S, Pammi M, Alston S, Khamees D, Peterson W, et al. 2021. Pivot to online learning for adapting or continuing workplace-based clinical learning in medical education following the COVID-19 pandemic: a BEME systematic review: BEME Guide No. 70. Med Teach. 23:1–17.
   Google Scholar
• Greenhalgh T, Vijayaraghavan S, Wherton J, Shaw S, Byrne E, Campbell-Richards D, Bhattacharya S, Hanson P, Ramoutar S, Gutteridge C, et al. 2016. Virtual online consultations: advantages and limitations (VOCAL) study. BMJ Open. 6(1):e009388.
   PubMed Web of Science ®Google Scholar
• Gulikers JTM, Bastiaens TJ, Kirschner PA. 2004. A five-dimensional framework for authentic assessment. ETR&D. 52(3):67–86.
   Web of Science ®Google Scholar
• Harley JM, Lou NM, Lui Y, Cutumisu M, Daniels LM, Leighton JP, Nadon L. 2021. University students’ negative emotions in a computer-based examination: the roles of trait test-emotion, prior test taking methods and gender. Assess Eval High Educ. 46(6):656–972.
   Web of Science ®Google Scholar
• Harrison A, Phelps M, Nerminathan A, Alexander S, Scott KM. 2019. Factors underlying students’ decisions to use mobile devices in clinical settings. Br J Educ Technol. 50(2):531–545.
   Web of Science ®Google Scholar
• Hart J. 2021. Top tools for learning 2021; [accessed 2022 Jan 21]. https://www.toptools4learning.com.
   Google Scholar
• Heeneman S, de Jong LH, Dawson LJ, Wilkinson TJ, Ryan A, Tait GR, Rice N, Torre D, Freeman A, van der Vleuten CPM. 2021. Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles. Med Teach. 43(10):1139–1148.
   PubMed Web of Science ®Google Scholar
• Hegazy NN, Elrafie NM, Saleh N, Youssry I, Ahmed SA, Yosef M, Ahmed MM, Rashwan NI, Abdel Malak HW, Girgis SA, et al. 2021. Consensus meeting report “technology enhanced assessment” in Covid-19 Time, MENA regional experiences and reflections. Adv Med Educ Pract. 12(12):1449–1456.
   PubMed Web of Science ®Google Scholar
• Hodges B. 2015. Sea monsters & whirlpools: navigating between examination and reflection in medical education. Med Teach. 37(3):261–266.
   PubMed Web of Science ®Google Scholar
• Hodges B. 2021. Performance-based assessment in the 21st century: when the examiner is a machine. Perspect Med Educ. 10(1):3–5.
   PubMed Web of Science ®Google Scholar
• Holmboe E. 2017. Work-based assessment and co-production in postgraduate medical training. GMS J Med Educ. 34(5):Doc58.
   PubMed Web of Science ®Google Scholar
• Homer M, Fuller R, Pell G. 2018. The benefits of sequential testing: Improved diagnostic accuracy and better outcomes for failing students. Med Teach. 40(3):275–284.
   PubMed Web of Science ®Google Scholar
• Hope D, Davids V, Bollington L, Maxwell S. 2021. Candidates undertaking (invigilated) assessment online show no differences in performance compared to those undertaking assessment offline. Med Teach. 43(6):646–650.
   PubMed Web of Science ®Google Scholar
• Hope D, Dewar A, Hothersall EJ, Leach JP, Cameron I, Jaap A. 2021. Measuring differential attainment: a longitudinal analysis of assessment results for 1512 medical students at four Scottish medical schools. BMJ Open. 11(9):e046056.
   PubMed Web of Science ®Google Scholar
• Hytönen H, Näpänkangas R, Karaharju-Suvanto T, Eväsoja T, Kallio A, Kokkari A, Tuononen T, Lahti S. 2021. Modification of national OSCE due to COVID-19 – implementation and students' feedback. Eur J Dent Educ. 25(4):679–688.
   PubMed Web of Science ®Google Scholar
• International Telecommunication Union. 2014. Measuring the Information Society. Cited in Fuller R, Joynes V. 2015. Enhancement or replacement? Understanding how legitimised use of mobile learning resources is shaping how healthcare students are learning. J EAHIL. 11(2):7–10.
   Google Scholar
• Jaap A, Dewar A, Duncan C, Fairhurst K, Hope D, Kluth D. 2021. Effect of remote online exam delivery on student experience and performance in applied knowledge tests. BMC Med Educ. 21(1):86.
   PubMed Web of Science ®Google Scholar
• JISC. 2016. The assessment and feedback lifecycle; [accessed 2022 Jan 21]. https://www.jisc.ac.uk/guides/transforming-assessment-and-feedback/lifecycle.
   Google Scholar
• JISC. 2020. Developing mental health and well being technologies and analytics; [accessed 2022 Jan 21]. https://www.jisc.ac.uk/rd/projects/developing-mental-health-and-wellbeing-technologies-and-analytics.
   Google Scholar
• Johanns B, Dinkens A, Moore J. 2017. A systematic review comparing open-book and closed-book examinations: evaluating effects on development of critical thinking skills. 2017. Nurse Educ Pract. 27:89–94.
   PubMed Web of Science ®Google Scholar
• Jones C, Ramanau R, Cross S, Healing G. 2010. Net generation of digital natives: is there a distinct new generation entering university? Comput Educ. 54(3):722–732.
   Web of Science ®Google Scholar
• Jones C. 2010. A new generation of learners? The net generation and digital natives. Learn Media Technol. 35(4):365–368.
   Web of Science ®Google Scholar
• Joynes V, Fuller R. 2016. Legitimisation, personalisation and maturation: using the experiences of a compulsory mobile curriculum to reconceptualise mobile learning. Med Teach. 38(6):621–627.
   PubMed Web of Science ®Google Scholar
• Judd T, Ryan A, Flynn E, McColl G. 2017. If at first you don't succeed… adoption of iPad marking for high-stakes assessments. Perspect Med Educ. 6(5):356–361.
   PubMed Web of Science ®Google Scholar
• Khalaf K, El-Kishawi MA, Al Kawas S. 2020. Introducing a comprehensive high-stake online exam to final-year dental students during the COVID-19 pandemic and evaluation of its effectiveness. Med Educ Online. 25:1.
   Web of Science ®Google Scholar
• Kickert R, Meeuwisse M, Stegers-Jager KM, Prinzie P, Arends LR. 2022. Curricular fit perspective on motivation in higher education. High Educ. 83(4):729–745.
   Web of Science ®Google Scholar
• Konings KD, van Berlo J, Koopmans R, Hoogland H, Spanjers IAE, ten Haaf JA, van der Vleuten CPM, van Merrienboer JJG. 2016. Using a smartphone app and coaching group sessions to promote residents’ reflection in the workplace. Acad Med. 91(3):365–370.
   PubMed Web of Science ®Google Scholar
• Korhonen V. 2021. Academically engaged or not? An exploratory study on modelling and assessing the engagement of first-year students at a research-intensive university in Finland. Assess Eval High Educ. 46(7):1048–1064.
   Web of Science ®Google Scholar
• Lai H, Gierl MJ, Touchie C, Pugh D, Boulais AP, De Champlain A. 2016. Using automatic item generation to improve the quality of MCQ distractors. Teach Learn Med. 28(2):166–173.
   PubMed Web of Science ®Google Scholar
• Lawson C, Beer C, Rossi D, Moore T, Fleming J. 2016. Identification of ‘at risk’ students using learning analytics: the ethical dilemmas of intervention strategies in a higher education institution. Educ Tech Res Dev. 64(5):957–968.
   Web of Science ®Google Scholar
• Leonhard G. 2021. Digital transformation: are you ready for exponential change? (video); [accessed 2022]. https://www.futuristgerd.com.
   Google Scholar
• Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, et al. 2019. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 1(6):E271–E297.
   PubMedGoogle Scholar
• Malecka B, Boud D. 2021. Fostering student motivation and engagement with feedback through ipsative processes. Teach High Educ. DOI:10.1080/13562517.2021.1928061
   Web of Science ®Google Scholar
• Marzano R, Kendall J. 2007. The new taxonomy of educational objectives. 2nd ed. California (US): Corwin Press.
   Google Scholar
• Maudsley G, Taylor D, Allam O, Garner J, Calinici T, Linkman K. 2019. A best evidence medical education (BEME) systematic review of: what works best for health professions students using mobile (hand-held) devices for educational support on clinical placements? BEME Guide No. 52. Med Teach. 41(2):125–140.
   PubMed Web of Science ®Google Scholar
• McKinsey. 2022. The fourth industrial revolution will be people powered; [accessed 2022 Jan 22]. https://www.mckinsey.com/business-functions/operations/our-insights/the-fourth-industrial-revolution-will-be-people-powered.
   Google Scholar
• McMenamin L, Brown FE, Arora M, Barnard J, Smith LE, Stockell DJ, Tung P, Wakefield RJ, Weerasinghe A, Wolstenhulme S. 2021. Twelve tips for integrating ultrasound guided peripheral intravenous access clinical skills teaching into undergraduate medical education. Med Teach. 43(9):1010–1018.
   PubMed Web of Science ®Google Scholar
• Mooney JS, Cappelli T, Byrne-Davis L, Lumsden CJ. 2014. How we developed eForms: an electronic form and data capture tool to support assessment in mobile medical education. Med Teach. 36(12):1032–1037.
   PubMed Web of Science ®Google Scholar
• Mozilla. 2017. Digital inclusion. Internet Health Issue Briefs; [accessed 2022 Jan 21]. https://assets.mozilla.net/pdf/IHPbriefs_Digital_Inclusion_March_2017.pdf.
   Google Scholar
• NHS eportfolios. 2021. [accessed 2022 Jan 21]. https://www.nhseportfolios.org/Information/InfoPage?page=WhoUsesEPortfolio.
   Google Scholar
• Norcini JM, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Mackay MFP, Roberts T, Swanson D. 2018. 2018 Consensus framework for good assessment. Med Teach. 40(11):1102–1109.
   PubMed Web of Science ®Google Scholar
• OECD. 2015. OECD future of education and skills 2030; [accessed 2022 Jan 21]. https://www.oecd.org/education/2030-project/.
   Google Scholar
• Oudkerk Pool A, Jaarsma ADC, Driessen EW, Govaerts MJB. 2020. Student perspectives on competency-based portfolios: does a portfolio reflect their competence development? Perspect Med Educ. 9(3):166–172.
   PubMed Web of Science ®Google Scholar
• Pack R, Lingard L, Watling C, Cristancho S. 2020. Beyond summative decision making: illuminating the broader roles of competence committees. Med Educ. 54(6):517–527.
   PubMed Web of Science ®Google Scholar
• Pell G, Fuller R, Homer M, Roberts T. 2012. Is short-term remediation after OSCE failure sustained? A retrospective analysis of the longitudinal attainment of underperforming students in OSCE assessments. Med Teach. 34(2):146–150.
   PubMed Web of Science ®Google Scholar
• Pell G, Fuller R, Homer M, Roberts T. 2013. Advancing the objective structured clinical examination: sequential testing in theory and practice. Med Educ. 47(6):569–577.
   PubMed Web of Science ®Google Scholar
• Philips. 2021. Three reasons why healthcare providers are moving to the cloud; [accessed 2022 Jan 21]. https://www.philips.com/a-w/about/news/archive/blogs/innovation-matters/2021/20210803-three-reasons-why-healthcare-providers-are-moving-to-the-cloud.html.
   Google Scholar
• Pickering J, Joynes V. 2016. A holistic model for evaluating the impact of individual technology-enhanced learning resources. Med Teach. 38(12):1242–1247.
   PubMed Web of Science ®Google Scholar
• Prensky M. 2001. Digital natives, digital immigrants. Horizon. 9:1–6.
   Google Scholar
• Prochazkova K, Novotny P, Hancarova M, Prchalova D, Sedlacek Z. 2019. Teaching a difficult topic using a problem-based concept resembling a computer game: development and evaluation of an e-learning application for medical molecular genetics. BMC Med Educ. 19(1):390.
   PubMed Web of Science ®Google Scholar
• Pyörälä E, Mäenpää S, Heinonen L, Folger D, Masalin T, Hervonen H. 2019. The art of note taking with mobile devices in medical education. BMC Med Educ. 19(1):96.
   PubMed Web of Science ®Google Scholar
• Rahim AFA. 2020. Guidelines for online assessment in emergency remote teaching during the COVID-19 pandemic. Educ Med J. 12(2):59–68.
   Google Scholar
• Rajpurkar P, Chen E, Banerjee O, Topol EJ. 2022. AI in health and medicine. Nat Med. 28(1):31–38.
   PubMed Web of Science ®Google Scholar
• Ricketts C, Bligh J. 2011. Developing a “frequent look and rapid remediation” assessment system for a new medical school. Acad Med. 86(1):67–71.
   PubMed Web of Science ®Google Scholar
• Roberts T, Samarasekera DD, Gibbs T, Sherman L. 2020. With Covid-19: assessment, professionalism and progression. Part 2 – maintaining professionalism during on-line assessment. AMEE; [accessed 2022 Jan 21]. https://amee.org/webinars/archive/2019-(1).
   Google Scholar
• Roman P, Ruiz-Gonzalez C, Rodriguez-Arrastia M, Granero-Molina J, Fernández-Sola C, Hernández-Padilla JM. 2022. A serious game for online-based objective structured clinical examination in nursing: a qualitative study. Nurse Educ Today. 109:105246.
   PubMed Web of Science ®Google Scholar
• Sadler DR. 2010. Beyond feedback: developing student capability in complex appraisal. Assess Eval High Educ. 35(5):535–550.
   Web of Science ®Google Scholar
• Sam A, Field SM, Collares CF, van der Vleuten CPM, Wass VJ, Melville C, Harris J, Meeran K. 2018. Very-short-answer questions: reliability, discrimination and acceptability. Med Educ. 52(4):447–455.
   PubMed Web of Science ®Google Scholar
• Sambell K, Brown S. 2021. Covid-19 assessment collection; [accessed 2022 Jan 21]. https://sally-brown.net/kay-sambell-and-sally-brown-covid-19-assessment-collection/.
   Google Scholar
• Scott KM, Nerminathan A, Alexander S, Phelps M, Harrison A. 2017. Mobile devices for learning in clinical settings. Br J Educ Technol. 48(1):176–190. 10.1111/bjet.12352.
   Web of Science ®Google Scholar
• Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X. 2021. A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Aust. 1329878X2110058.
   Web of Science ®Google Scholar
• Selwyn N. 2010. Looking beyond learning: notes towards the critical study of educational technology. J Comput Assist Learn. 26(1):65–73.
   Web of Science ®Google Scholar
• Selwyn N. 2021. ‘AI’ in Education: two sides of a conversation. Critical Studies of Education & Technology; [accessed 2022 Jan 21]. https://criticaledtech.com/2021/11/26/ai-in-education-two-sides-of-a-conversation/.
   Google Scholar
• Shum SJB, Luckin R. 2019. Learning analytics and AI: politics, pedagogy and practices. Br J Educ Technol. 50(6):2785–2793.
   Web of Science ®Google Scholar
• Silveira JM. 2013. Idea Bank: portfolios and assessment in music classes. Music Educ J. 99(3):15–24.
   Google Scholar
• Sonderlund AL, Hughes E, Smith J. 2019. The efficacy of learning analytics interventions in higher education: a systematic review. Br J Educ Technol. 50(5):2594–2618.
   Web of Science ®Google Scholar
• Spiegel T, Nivette A. 2021. The relative impact of in-class closed-book versus take-home open-book examination type on academic performance, student knowledge retention and wellbeing. Assess Eval High Educ. DOI:10.1080/02602938.2021.2016607
   Web of Science ®Google Scholar
• St-Onge C, Young M, Eva KW, Hodges B. 2017. Validity: one word with a plurality of meanings. Adv Health Sci Educ Theory Pract. 22(4):853–867.
   PubMed Web of Science ®Google Scholar
• Sundgren M. 2017. Blurring time and place in higher education with bring your own device applications: a literature review. Educ Inf Technol. 22(6):3081–3119.
   Web of Science ®Google Scholar
• Tapscott D. 1998. Growing up digital: the rise of the net generation. New York, NY: McGraw‐Hill.
   Google Scholar
• Torre D, Rice NE, Ryan A, Bok H, Dawson LJ, Bierer B, Wilkinson TJ, Tait GR, Laughlin T, Veerapen K, et al. 2021. Ottawa 2020 consensus statements for programmatic assessment – 2. Implementation and practice. Med Teach. 43(10):1149–1160.
   PubMed Web of Science ®Google Scholar
• Tran VT, Ravaud P. 2016. Frugal innovation in medicine for low resource settings. BMC Med. 14(1):102.
   PubMed Web of Science ®Google Scholar
• Tsai YS, Perrotta C, Gašević D. 2020. Empowering learners with personalised learning approaches? Agency, equity and transparency in the context of learning analytics. Assess Eval High Educ. 45(4):554–567.
   Web of Science ®Google Scholar
• Tsoy D, Sneath P, Rempel J, Simon H, Bodnariuc N, Mercuri M, Alim P, Chan T. 2019. Creating GridlockED: a serious game for teaching about multipatient environments. Acad Med. 94(1):66–70.
   PubMed Web of Science ®Google Scholar
• Tuti T, Winters N, Edgcombe H, Muinga N, Wanyama C, English M, Paton C. 2020. Evaluation of adaptive feedback in a smartphone based game on health care providers’ learning gain: randomized control trial. J Med Internet Res. 22(7):e17100.
   PubMed Web of Science ®Google Scholar
• Tweed M, Desrosiers J, Wilkinson TJ. 2021. Randomised controlled trial of students access to resources in an examination. Med Educ. 55(8):951–960.
   PubMed Web of Science ®Google Scholar
• UNESCO. 2018. Defining internet universality indicators. https://en.unesco.org/[email protected].
   Google Scholar
• van der Schaaf M, Donkers J, Slof B, Moonen-van Loon J, van Tartwijk J, Driessen E, Badii A, Serban O, en Cate O. 2017. Improving workplace-based assessment and feedback by an E-portfolio enhanced with learning analytics. Educ Tech Res Dev. 65(2):359–380.
   Web of Science ®Google Scholar
• Villarroel V, Bloxham S, Bruna D, Bruna C, Herrera-Seda C. 2018. Authentic assessment: creating a blueprint for course design. Assess Eval High Educ. 43(5):840–854.
   Web of Science ®Google Scholar
• Villarroel V, Boud D, Bloxham S, Bruna D, Bruna C. 2019. Using principles of authentic assessment to redesign written examinations and tests. Innov Educ Teach Int. 57(1):1–49.
   Web of Science ®Google Scholar
• Wang RDA, DeMaria S, Goldberg A, Katz D. 2016. A systematic review of serious games in training health care professionals. Simul Healthc. 11(1):41–51.
   PubMedGoogle Scholar
• Westerkamp AC, Heijne-Penninga M, Kuks JBM, Cohen-Schotanus J. 2013. Open-book tests: search behaviour, time used and test scores. Med Teach. 35(4):330–332.
   PubMed Web of Science ®Google Scholar
• Wiliam D. 2021. Improving medical education. Prospects and challenges. Association of Medical Education in Europe Conference Plenary.
   Google Scholar
• Wilkinson T, Nadarajah VD. 2021. Assuring equivalence of assessments across settings. Understanding assessment in medical education through quality assurance. Chapter 8. McGraw Hill; p. 190–215.
   Google Scholar
• Winston KA, van der Vleuten CPM, Scherpbier AJJA. 2012. The role of the teacher in remediating at-risk medical students. Med Teach. 34(11):e732–e742.
   PubMed Web of Science ®Google Scholar
• Wood GC. 2020a. Preparing for take-home and open book exams: a student guide; [accessed 2022 Jan 21]. https://www.garycwood.uk/2020/05/preparing-for-take-home-and-open-book-exams.html.
   Google Scholar
• Wood GC. 2020b. Sitting your take-home and open book exams: a student guide; [accessed 2022 Jan 21]. https://www.garycwood.uk/2020/05/sitting-your-take-home-and-open-book-exams.html.
   Google Scholar
• Yan BP, Lai WHS, Chan CKY, Au ACK, Freedman B, Poh YC, Poh MZ. 2020. High-throughput, contact-free detection of atrial fibrillation from video with deep learning. JAMA Cardiol. 5(1):105–107.
   PubMed Web of Science ®Google Scholar
• Yeates P, McCray G, Moult A, Cope N, Fuller R, McKinley R. 2022. Determining the influence of different linking patterns on the stability of students’ score adjustments produced using Video-based examiner score comparison and adjustment (VESCA). BMC Med Educ. 22 41. DOI:10.1186/s12909-022-03115-1
   PubMed Web of Science ®Google Scholar
• Yeates P, Moult A, Cope N, McCray G, Fuller R, McKinley R. 2021a. Determining influence, interaction and causality of contrast and sequence effects in objective structured clinical exams. Med Educ. 56(3):292–302.
   Web of Science ®Google Scholar
• Yeates P, Moult A, Cope N, McCray G, Xilas E, Lovelock T, Vaughan N, Daw D, Fuller R, McKinley RK. 2021b. Measuring the effect of examiner variability in a multiple-circuit objective structured clinical examination (OSCE). Acad Med. 96(8):1189–1196.
   PubMed Web of Science ®Google Scholar
• Yeates P, Moult A, Lefroy J, Walsh-House J, Clews L, McKinley R, Fuller R. 2020. Understanding and developing procedures for video-based assessment in medical education. Med Teach. 42(11):1250–1260.
   PubMed Web of Science ®Google Scholar
• Zagury-Orly I, Durning SJ. 2021. Assessing open-book examination in medical education: the time is now. Med Teach. 43(8):972–973.
   PubMed Web of Science ®Google Scholar