Abstract
Introduction
Programmatic assessment is an approach to assessment aimed at optimizing the learning and decision function of assessment. It involves a set of key principles and ground rules that are important for its design and implementation. However, despite its intuitive appeal, its implementation remains a challenge. The purpose of this paper is to gain a better understanding of the factors that affect the implementation process of programmatic assessment and how specific implementation challenges are managed across different programs.
Methods
An explanatory multiple case (collective) approach was used for this study. We identified 6 medical programs that had implemented programmatic assessment with variation regarding health profession disciplines, level of education and geographic location. We conducted interviews with a key faculty member from each of the programs and analyzed the data using inductive thematic analysis.
Results
We identified two major factors in managing the challenges and complexity of the implementation process: knowledge brokers and a strategic opportunistic approach. Knowledge brokers were the people who drove and designed the implementation process acting by translating evidence into practice allowing for real-time management of the complex processes of implementation. These knowledge brokers used a ‘strategic opportunistic’ or agile approach to recognize new opportunities, secure leadership support, adapt to the context and take advantage of the unexpected. Engaging in an overall curriculum reform process was a critical factor for a successful implementation of programmatic assessment.
Discussion
The study contributes to the understanding of the intricacies of implementation processes of programmatic assessment across different institutions. Managing opportunities, adaptive planning, awareness of context, were all critical aspects of thinking strategically and opportunistically in the implementation of programmatic assessment. Future research is needed to provide a more in-depth understanding of values and beliefs that underpin the assessment culture of an organization, and how such values may affect implementation.
Disclosure statement
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.
Glossary
Programmatic assessment: Is an approach to assessment aimed at optimizing the learning and decision function of assessment.
Additional information
Notes on contributors
Dario Torre
Dario Torre, MD, MPH, PHD, is professor of Medicine and Director of programs of assessment at University of Central Florida College of Medicine.
Lambert Schuwirth
Lambert Schuwirth, MD, PHD, is professor of medicine at Flinders University College of Medicine and Public Health, Adelaide, Australia.
Cees Van der Vleuten
Cees Van der Vleuten, PHD, is Professor of Education Department of Educational Development and Research, School of Health Profession Education, Maastricht University, The Netherlands.
Sylvia Heeneman
Sylvia Heeneman, PHD, is Professor of Education Department of Pathology, School Health Profession Education, Cardiovascular Research Institute Maastricht, Maastricht University, The Netherlands.