Teachers’ conceptions of assessment literacy

ABSTRACT

Teachers’ assessment literacy affects the quality of assessments and is, therefore, an essential part of teachers’ competence. Recent studies define assessment literacy as a dynamic, contextual and social construct, situated in practice and mediated by teachers’ identity and conceptions of assessment. This study provides a further elaboration of assessment literacy by exploring teachers’ conceptions of assessment literacy from a sociocultural perspective. Eleven online focus group interviews were conducted within the context of Dutch higher professional education between June and December 2020. A template analysis method was used to analyse the data. Seven interrelated aspects of assessment literacy were identified, namely ‘continuously developing assessment literacy’, ‘conscientious decision making’, ‘aligning’, ‘collaborating’, ‘discussing’, ‘improving and innovating’, and ‘coping with tensions’. This representation of assessment literacy, based on teachers’ conceptions, may guide teachers’ development of assessment literacy in practice.

KEYWORDS:

• Teachers’ assessment literacy
• teachers’ professional development
• higher professional education
• focus groups
• template analysis

Introduction

Teachers’ assessment literacy (AL) affects the quality of assessments and is, therefore, an essential part of teachers’ competence (Popham, 2009). High-quality assessments inform teachers about students’ learning processes, support teachers’ decision-making, and motivate students to improve their performance (Andrade & Brookhart, 2016). In addition, the quality of assessments determines the value of students’ diplomas (Brookhart, 2011; Lees & Anderson, 2015; Popham, 2009). Therefore, teachers’ AL is intrinsic to teachers’ competence. Teachers’ AL is often defined as a combination of knowledge and skills, and is generally operationalised as standards of assessment knowledge and skills, measured in questionnaires (Mertler, 2004; Xu & Brown, 2017).

In recent years, however, the definition of AL as a fixed set of knowledge and skills has been criticised (Deneen & Boud, 2014; Deneen et al., 2016; Willis et al., 2013; Xu & Brown, 2016). In contemporary literature on teachers’ competence and continuous professional development, knowledge and skills are seen as embedded in daily practice and, therefore, knowledge and skills are viewed as inherently social and authentic in nature (Eraut, 2004; Kyndt et al., 2016). To this end, several recent studies describe AL in terms of a dynamic, contextual and social construct, situated in practice and mediated by teachers’ identity and conceptions of assessment (Boud et al., 2018; Pastore & Andrade, 2019; Xu & Brown, 2016).

This new perspective on AL, mainly based on existing literature (Looney et al., 2017; Xu & Brown, 2016) and supplemented with expert opinions (Pastore & Andrade, 2019), provides little insight into teachers’ conceptions of AL. These aforementioned scholars suggest exploring teachers’ conceptions of AL, because teachers’ conceptions influence behaviour and steer actions, and consequently have a significant impact on the enactment of AL in practice (Pastore & Andrade, 2019; Xu & Brown, 2016). Therefore, the purpose of this study is to further elucidate the concept of AL by studying teachers’ conceptions of AL from this new perspective: a dynamic, contextual, and social process.

Theoretical framework

Traditional definitions of AL tend to focus on gathering objective data about teachers’ AL and their professional development (Boud et al., 2018; DeLuca & Johnson, 2017; Willis et al., 2013). Reliability and construct validity are seen as important criteria of teachers’ AL to ensure that outcomes are reproducible and independent of specific situations (DeLuca & Johnson, 2017; Schuwirth & van der Vleuten, 2019). In this tradition, teachers’ AL mainly focusses on summative functions of assessments and is listed as a fixed set of standards, applicable to all teachers in all situations (American Federation of Teachers et al., 1990; Brookhart, 2011). These standards are used by organisations for the design of professionalisation activities and to make decisions about teachers’ AL.

However, these traditional definitions of AL are not congruent with the way we currently define professional competence, i.e. as the ability to perform in a certain job or task, using embedded capabilities in a distinct organisational context and specific situations. Teachers’ competence is thus not universal, certain or fixed (Caena, 2011; Mulder, 2014). Therefore, several scholars suggest to reconsider the traditional definitions of AL (Looney et al., 2017; Pastore & Andrade, 2019; Willis et al., 2013; Xu & Brown, 2016). Taking a sociocultural perspective, they suggest that AL is a dynamic, developing process, mediated by interactions with other people, by the culture teachers work in, and affected by deeply rooted beliefs and attitudes. From this perspective, teachers’ AL is continuously reconstructed over time as a result of a variety of external, contextual and internal forces (Looney et al., 2017; Willis et al., 2013; Xu & Brown, 2016).

The close association of teachers’ AL with assumptions about learning and the embeddedness of AL in the cultural and policy context is underpinned by Willis et al. (2013). They describe AL as ‘dynamic social practices which are context-dependent and which involve teachers in articulating and negotiating classroom and cultural knowledges with one another and with learners, in the initiation, development, and practice of assessment to achieve the learning goals of students’ (p1). AL is further described by Xu and Brown (2016) as a competence that combines formal knowledge about assessment with conceptions of assessment that enables teachers to deal with assessment issues within the contexts in which assessments take place. The influence of these contexts is emphasised by the fact that AL is constrained by assessment policies, rules and regulations. Furthermore, a knowledge base of assessment concepts is a necessary prerequisite to resolve complex and diverse assessment issues (Xu & Brown, 2016). Such a knowledge base is filtered and mediated by teachers’ conceptions. Hence, the uptake and enactment of AL in practice is influenced by teachers’ conceptions of AL.

Teachers’ AL goes beyond knowledge and skills, and includes self-confidence, self-efficacy as well as beliefs and emotions. Teachers’ conceptions of assessment, attitudes during assessments and perceptions about their role as an examiner are aspects of teachers’ identity (Looney et al., 2017). Accordingly, AL develops and redevelops during the teaching career and is shaped by different situations in which the teacher works and lives.

Pastore and Andrade (2019) integrated the above-mentioned viewpoints on AL in three key dimensions. Firstly, the dimension of conceptual knowledge includes teachers’ conceptions of assessment, teaching and learning. Secondly, the praxeological dimension positions AL in practice where teachers integrate assessment processes and where teachers have to act on multiple and sometimes conflicting demands. Lastly, the socio-emotional dimension includes teachers’ awareness of their position as an examiner, as well as ethical aspects and awareness of the impact of assessing.

Furthermore, AL should be described as activities embedded in practice, needed for effective performance in a certain profession, organisation or role. In these activities knowledge, skills and attitudes are integrated (Mulder, 2014). Given that teachers’ AL is a constant negotiation within specific contexts, embedded in practice, and mediated by teachers’ conceptions and experiences, we argue that teachers’ AL should be seen as adaptive and continuously developing.

From a socio-cultural perspective, there is thus a theoretical understanding of AL, namely that it is a dynamic, contextual and social construct. However, how this concept of AL is understood by teachers is still underexplored. In this study, we explored teachers’ conceptions of AL since these conceptions have impact on their AL in practice. We consider teachers’ conceptions inclusive of perceptions, opinions, attitudes and other terms that suggest their beliefs about AL (De Bruijn, 2012). To this end, the following research question was articulated: how do teachers conceive of AL as being a dynamic, contextual and social construct?

Materials and methods

Context

This study was conducted within the context of Dutch higher professional education (HPE) offered by universities of applied sciences (UASs). The levels of HPE correspond with Levels 5–7 of the International Standard Classification of Education (ISCED) and the European Qualification Framework (EQF). HPE teachers can be seen as career switchers, transitioning from their original vocation, such as physical therapists or journalists, to teacher. Dutch HPE teachers often do not have a teaching degree, so the universities offer professionalisation activities on AL. In general, these activities are based on traditional definitions of AL and are perceived by HPE teachers as detached from their assessment practice (Webb, 2002; Wubbels & Tartwijk Van, 2019). For their role as an examiner, Dutch HPE teachers must obtain a certificate. Some HPE teachers have transcendent roles, besides the role as an examiner, for instance as members of an examination board or assessment committee. Or they are, for instance, involved in designing and implementing assessment programmes, which contain multiple related assessments.

Method

We assumed it would be challenging to elicit teachers’ conceptions of AL as being a dynamic, contextual, and social construct because Dutch HPE teachers are generally matured with traditional AL definitions. Nonetheless, we expected that HPE teachers would be more or less familiar with the ideas of fostering continuous professional development as they try to stimulate this in their students. To gain an in-depth understanding of AL, we adopted a qualitative approach. A focus group method was chosen to challenge participants to explicate their conceptions of AL in dialogues and to elicit tacit perspectives (Kitzinger, 2005).

Participants

Participants were experienced HPE teachers (minimum of three years of teaching experience). They were recruited by key figures, who have a prominent position in their UAS regarding professionalisation activities on teaching competence. These key figures represent their university in a national network of UASs to improve the professionalisation of HPE teachers. The key figures invited teachers from within their university to participate, resulting in a total of 44 participating HPE teachers from fifteen UASs. Table 1 presents the participants’ demographic characteristics.

Table 1. Characteristics of the participants.

	Participants
Total (n)	44
Number of UASs	15
Gender	16 male, 28 female
Years of experience	Median 10 (R3–36) Average 12*
Roles participants	15 examination board, 10 assessment committee, 7 curriculum designer, 2 assessment expert
Assessment Qualifications	2 none, 32 basic, 17 senior, 5 master of education

*Five higher professional education teachers had 10 to 28 years of previous experience in education in secondary, higher, or academic education.

Procedure

Eleven focus groups were held online because of the COVID-19 lockdown measures between June and December 2020. To foster discussions in an online environment, the group size was three to six participants (Woodyatt et al., 2016). To ensure a sample size large enough to find patterns and themes, the total of eleven focus groups was above average (Nyumba et al., 2018). A mixed composition of teachers from different UASs within the groups was chosen to encourage discussions (Onwuegbuzie & Collins, 2007). The focus groups sessions were recorded and transcribed verbatim. The participants were numbered to refer to the focus group interview transcripts in such a manner that the used quotes remained retrievable. The study was approved by the Research Ethics Committee (cETO) of the Open University of the Netherlands (no. U202003589).

Instruments

A vignette strategy was used to challenge the participants to adopt a sociocultural perspective and go beyond traditional definitions of AL. Besides, the use of vignettes supports equal contributions of the participants in the discussion, because participants are asked to share their conceptions about the vignette before starting the discussion (Auspurg et al., 2015). Vignettes are carefully constructed short, hypothetical though realistic scenarios (Stravakou & Lozgka, 2018). An open way of constructing vignettes was used in this study to fit the study’s exploratory nature (Auspurg et al., 2015). Three short authentic scenarios were composed based on the main aspects of our theoretical framework: AL as a dynamic, contextual and social construct. The first vignette described a situation of a teacher adapting to changes in his assessment practice to address the dynamic aspects of AL. The second vignette described a situation of a teacher acting in two different educational programmes to address the contextual aspects of AL. The last vignette described a situation of a teacher interacting with his social network on assessments to address the social aspects of AL. The vignettes were constructed by the researcher and tested in a pilot focus group to determine if they were transparent and supporting discussions.

Following an interview guide, the focus groups started with an open question about how the participants conceive AL as an ongoing process. Then, the participants were asked to comment on the first scenario. Participants were given time to consider their initial thoughts. Then, the scenario was discussed, using the interview guide which included sub-questions for the researcher to encourage discussion if necessary. For instance, on how an assessment literate teacher might react. This process was repeated for the second and third scenario. Towards the end of the interview, participants were asked to summarise their conceptions of AL.

Data analysis

Data were analysed using a template analysis method to develop a hierarchical organisation of codes identified in the data (Brooks et al., 2015). In a step-by-step iterative process, following the procedural steps by Brooks et al (Brooks et al., 2015), meaningful text fragments of the transcribed focus groups were coded, organised, evaluated and adjusted to develop the template. The developmental process of the template is shown in Figure 1. Each step of this process was systematically and repeatedly discussed within the research group (i.e. all authors) until consensus was reached.

Figure 1. Developmental process of the template.

In the first phase of template analysis, we used three a priori themes to become familiar with the data based on our theoretical framework, namely AL as a dynamic, contextual and social construct. A sample of quotations was discussed with the second author to establish a first template. In the second phase, the first author inductively open-coded text fragments of the first six focus groups, combining antecedents and similar text fragments. The codes were applied to the next five focus groups and adjusted and supplemented, resulting in 37 codes. The third phase was a process of meaning making by the four authors collaboratively. The codes were organised into meaningful clusters based on the first template. The template was revised and modified in an iterative process, exploring the clusters, codes and text fragments backwards and forwards. Meaningful clusters were interpreted as aspects of AL. This phase resulted in the description of seven aspects of AL. In the fourth and last phase, the relations between the codes and seven identified aspects were further deepened. These relations were used to develop a visual representation of teachers’ conceptions of AL.

Results

In general, the participants were inclined to express their conceptions of AL in terms of what teachers do NOT do in practice and often talked about AL as aspirations. Although there were some indications that a sociocultural perspective on AL is not yet fully embedded in HPE teachers’ daily practice, we were able to allocate such expressions to meaningful aspects of AL. Seven aspects were identified, which together represent how teachers conceive of AL.

Continuously developing AL

The participants conceive AL as an ongoing process. Teachers are seen as assessment literate if they accept that their AL is never completed, as was expressed by participant 2.2:

“So, you have to keep developing. And you will never reach your hundred per cent. That is impossible. You can gather new things, incorporate them.”

This also means that AL includes gaining experiences with assessment and having some regularity in applying AL in practice. The participants speculated on assessment knowledge and skills as a prerequisite for developing AL. In addition, continuously developing AL is understood as a fundamental attitude. The participants used general terms to describe attitudinal aspects such as being reflective, critical and curious. AL is conceived as broader than assessing students, including reflection on teachers’ roles and responsibilities. They believe that AL involves connecting theory and practice and moving back and forth between various assessment concepts and assessment-in-practice:

“For me, it also means to have a complete overview and connecting fundamental theories and models with the educational practice and vice versa. It is seldom exactly corresponding with how it plays out in practice.” (Participant 3.2)

In sum, the aspect ‘continuously developing AL’ means that teachers view AL as a continuous process, including a knowledge base, attitudinal aspects, and moving back and forth between theory and ongoing practical experiences.

Conscientious decision making

The participants conceive assessment literate teachers as being conscientious in making decisions in their assessment practice. Decisions encompass judgemental decisions about student performances, decisions about assessment programmes or the assessment organisation, and decisions about possible assessment innovations. Assessment literate teachers do not go along with every hype, but reconsider the implications of changes for their assessment practice as pointed out by participant 7.2:

“You cannot, just like that, change from a knowledge test to an integral assessment because it’s a hype. Well, yes, there is a lot more to it.”

The participants added that decisions are more conscientious when they are based on multiple arguments. Assessment literate teachers can account for their decisions as participant 6.1 said:

“I would expect if they are assessment literate, that they search for evidence and examples to substantiate their point of view.”

The participants did not differentiate between decisions related to assessments of learning (summative) and decisions related to assessments for learning (formative). They conceived summative and formative functions of assessment as interrelated. The participants believe that AL involves being aware of the teacher’s role in making decisions, referring to multiple roles HPE teachers can have, for instance, member of an examination board besides examiner of a course. Accordingly, the aspect of AL ‘conscientious decisions making’ concerns a well considered substantiation of all kinds of assessment-related decisions.

Aligning

The participants conceive AL as aligning assessment practices with the educational context and the vocation the educational programme focuses on. They recognise differences in assessment practices between educational programmes, relating to different vocational cultures and student characteristics. AL is about aligning assessment practices with teaching in these specific contexts (participant 5.2):

“In my opinion, assessment literate teachers with a role as an examiner, are aware of things like alignment. So, they know the learning objectives, they know the assessments and think about the connection of this assessment with the design of their course and lessons.”

The participants believe that assessment literate teachers also align their assessment practice with assessment policy and procedures. Being assessment literate means acting within rules, regulations, and agreements, for example, the teaching and examination regulations. The participants think that HPE teachers often lack an overview of the entire curriculum and the world of work because they tend to focus on their subjects of expertise. In these situations, participants notice fragmented approaches and discord in the assessment practice. In sum, assessment literate teachers see assessing as an integral aspect of their teaching activities and not as a final or separate activity as illustrated by participant 8.1:

“The specific subject of the teachers’ expertise predominates AL. So I would like to see assessing as an integral part of the daily practice by everyone.”

Collaborating

The participants conceive AL as collaborating with various stakeholders to achieve common goals in assessment practices. They mention the importance of a shared vision of assessing and learning. Assessment literate teachers can adjust their ideas and opinions to shared views and compromises. Participant 6.1 summarised a discussion about this topic as follows:

“It is important to propagate a shared vision from the perspective of the educational programme, regardless of whether you really agree with it.”

This also became apparent in discussions about calibration sessions to ensure equitability and accuracy of performance appraisal ratings. Assessment literate teachers can adjust their personal standards and collaboratively develop shared standards. Assessment literate teachers do not have to be 5-legged-sheep, but can bring their own share when collaborating. Participant 6.2 described the importance of complementing each other as:

“Even if you have a senior certification in examination or if you are a member of the examination board, you probably do not know everything yourself.”

They discussed to which extent AL can be seen as part of the competence of a single teacher, or should be seen as a joint competence of a team. In that sense, AL means that teachers work on shared understandings and know who to turn to in their team.

Discussing

The participants conceive AL as discussing assessment issues, in which assessment literate teachers can take different perspectives. In contrast to collaborating, this aspect of AL focuses on the fact that assessment literate teachers get inspired by new ideas and different viewpoints. An illustration is given by participant 1.3:

“Of course, opinions and ideas differ. That is not bad at all. However, it is important to consult each other and listen to each other, so that you think; “Oh yes, well”, or maybe someone does not share your opinion. That, in itself, does not matter to me. The result should be that you are inspired by one another.”

The participants believe that taking part in discussions helps to explore the opinions of others and new possibilities. In addition, the participants think that sharing thoughts and opinions contributes to strengthening mutual trust, as is explicated by participant 7.1:

“Actually, I think, to gain trust and rely on colleagues is very important. Well, yes, we differ and we make mistakes along the way, from which we can learn.”

Improving and innovating

The participants conceive AL as improving and innovating the assessment practice. Assessment literate teachers evaluate their assessment practice so they can improve and adjust their assessment practice to changes in policy, work field and new (digital) developments in the assessment domain itself. Participants mentioned a plan-do-check-act cycle or an assessment cycle as methods for evaluation and improvement of assessment practice as mentioned by participant 11.4:

“What I just said about knowledge tests, is that an assessment literate teacher can use the test analysis as input for the next test. The evaluation helps to develop a certain competence in testing. This is the same with rubrics. You design rubrics beforehand and learn from the evaluation afterwards: “Were these good rubrics? Did this work? Did this lead to the right decisions we made?” And that can be used for refining the design of the rubrics next time. Like the PDCA [plan do check act] cycle.”

Hence, the participants emphasise that assessment literate teachers can adopt new ideas and other opinions about assessments to innovate and improve their assessment practice. Because the data collection took place during the COVID-19 pandemic, technological changes were frequently discussed.

Coping with tensions

The participants are apprehensive about AL and experience tensions when it comes to AL. They use words such as ‘dare’ and ‘courage’ in various quotations. The participants refer to different interests of stakeholders, societal importance, accountability issues, and the impact of decisions on students. They express their apprehension of AL in general terms. For example (participant 3.2):

“Assessing, for me, is such a major THING. I manage the teaching part; you get the hang of that in practice. But assessing, that becomes ‘wooh’ very important at once, very serious, or so.”

The participants often mention tensions that arise among two or more of the other aspects of AL, for instance, between innovating the assessment practice with new assessment forms that do not comply with existing regulations and policy. Participant 4.2 tells:

“When considering things like the reliability and validity of the assessment, that makes you conservative towards changes. I notice some reluctance myself sometimes, but I see this trend often when it concerns assessments.”

They argued that tensions negatively affect their professional development in AL. Thus, an assessment literate teacher can balance conflicting aspects within the assessment practice.

Visual representation of AL

The seven aspects of AL were identified as strongly interrelated, without sequence or hierarchy. The aspects are visualised as a pinwheel model (Figure 2).

Figure 2. Visualisation of teachers’ AL; a pinwheel model.

In this model, five aspects can be positioned as wings of the pinwheel. The aspect ‘continuously developing AL’ is visualised as fundamental to the other aspects and as the aspect that keeps the pinwheel in motion. The aspect ‘coping with tensions’ is positioned in the centre because tensions emerge among the aspects. Besides, the participants mentioned that tensions can hinder AL like a too-tight centre pin hinders a pinwheel to spin.

Due to the COVID-19 lockdown period, the need to make changes in the assessment practices was urgent, resulting in increased awareness of teachers’ AL and elucidating the relations between the aspects. For example, participants talked about assessment literate teachers as being able to make conscientious decisions about students using new technological surveillance methods, relating the aspect ‘conscientious decision making’ with the aspect ‘improving and innovating’. Furthermore, these two aspects were related to other aspects, for instance with the aspect ‘aligning’ by expressing their conceptions of AL as teachers who were able to align practical assignments with closed or online workplaces, or with the aspect ‘discussing’ by mentioning the numerous conversations about privacy and integrity issues with others.

Conclusion and discussion

To provide insight into how AL is understood by HPE teachers, this study focused on the question of how teachers conceive of AL from a socio-cultural perspective. Seven interrelated aspects of AL were derived from group interviews with HPE teachers. A pinwheel model was presented to visualise the interrelatedness between the aspects.

Comparing our representation of AL to previous research, a few findings warrant further discussion. First, fitting a socio-cultural perspective, AL is viewed as a social process, mediated by interactions with other people. The participants seem to recognise this perspective on AL, as the aspects ‘collaborating’ and ‘discussing’ refer specifically to assessment-related social activities, such as participating in calibration sessions. In many previous studies on AL (Lees & Anderson, 2015; Looney et al., 2017; Xu & Brown, 2016) social assessment-related activities are not explicitly mentioned. However, Pastore and Andrade (2019) presented similar social activities as part of their dimensions of AL, such as ‘working with colleagues, parents and other stakeholders’ as part of the socio-emotional dimension. The participants even mentioned viewing AL as a joint competence of a team. Such a point of view on AL might be an interesting angle for additional research.

Second, AL was considered a dynamic process. Therefore, the aspects ‘continuously developing AL’ and the aspect ‘coping with tensions’ were deliberately positioned differently compared to the other aspects. The aspect ‘continuously developing AL’ was positioned as underlying and fundamental to sustaining teachers’ AL since participants often referred to assessment literate teachers as being reflective professionals and being up-to-date. Studies on continuous professional development consider reflection as the cornerstone for development and characterise reflection as a prerequisite to identify behaviour or conditions that may need adjustment or improvement in the future (Evers et al., 2016). This aspect thus reinforces understanding AL in line with continuous professional development. The aspect ‘coping with tensions’ represents conflicts that occur in practice among aspects of AL. For instance, when teachers want to innovate the assessment practice, this might conflict with the aspect ‘aligning’ because innovations might not yet be included in the assessment policy. This aspect can be recognised in the work of Xu and Brown (2016), who describe teachers’ AL as ‘compromises made among tensions’ (p.157) in which quality assessments are seen as a temporary equilibrium reached among tensions. Although tensions are mostly seen by the participants as barriers to develop AL, learning to cope with tensions is frequently mentioned in studies to foster teachers’ professional development (Ng & Leicht, 2019; Schaap et al., 2019) and studies on pre-service teachers’ learning (Charteris & Dargusch, 2018). Hence, characteristics of teachers’ competence and continuous professional development also resonate in this aspect of AL.

Third, considering AL as contextual and embedded in practice, participants suggested that the aspects of AL were interrelated. This resembles the non-hierarchical, coherent model presented by Pastore and Andrade (2019), supporting the meaning AL has in a specific educational context. Previous studies on AL are predominantly focused on the role as examiner (Brookhart, 2011; Lees & Anderson, 2015; Looney et al., 2017; Xu & Brown, 2016). In the context of HPE, teachers have overarching roles and thus different priorities. The representation of AL in this study broadens AL to overarching roles, such as designing assessment programmes. This means that the seven aspects of AL can manifest differently in practice, depending on teachers’ roles. For instance, the aspect ‘making conscientious decisions’ can refer to student performance, but to assessment methods within a programme as well.

Last, assessment knowledge and skills are framed differently across different studies. On the one hand, Xu and Brown (2016) suggest a hierarchical structure with knowledge and skills as a basis for AL. Pastore and Andrade (2019) also present a dimension of AL called ‘conceptual knowledge’ to frame assessment knowledge and skills as part of AL. Some participants framed assessment knowledge akin as a prerequisite for AL. On the other hand, in literature on teachers’ competence, knowledge and skills are positioned as embedded in activities to effectively perform in teachers’ profession, tasks or roles (Caena, 2011; Mulder, 2014; Sturing et al., 2011). The majority of the participants in this study framed assessment knowledge and skills as being up-to-date to perform adequately in assessment-related situations. In order to align teachers’ AL with contemporary views of teachers’ competence, having up-to-date knowledge and skills was positioned within the aspect ‘continuously developing AL’.

Reflections on the methodology

An unanticipated change due to the COVID-19 pandemic forced teachers to adopt technology rapidly (Van der Spoel et al., 2020) and researchers to conduct interviews online. An advantage of the online setting was the diversity of the participants from different UASs and professional domains within each focus group. This diversity enabled participants to discuss differences in assessment cultures. A possible limitation of online focus groups is the lack of non-verbal communication (Dodds & Hess, 2021). Small group sizes enabled us to see everyone’s face, which made it easier to observe and react on non-verbal expressions.

Studying teachers’ conceptions might be criticised because the participants may tend to idealise AL (Mansour, 2009). Our methodological choice to use three scenarios counterbalanced the potential idealisation of AL. Besides, some participants themselves recognised potential idealisation by framing AL as a team effort. Most participants had transcending roles in assessing, such as member of an examination board, and were highly interested in assessment issues and AL. This means that the variety of HPE teachers’ roles was limited within the focus groups. Using a more stratified sampling strategy across teachers’ roles might have enhanced the diversity in HPE teachers’ roles. Further research is needed to answer the question whether the aspects of AL identified in the current study are also recognised by the majority of HPE teachers, who do not have transcending roles.

The template analysis method allowed us to adopt our theoretical framework and suited our research question as a highly flexible approach. The traceability of the iterative processes in developing the template, however, is less secure than in other methods like thematic analysis (Brooks et al., 2015). The consensus meetings with all authors supported the traceability and accountability of the development of the framework.

Implications

This study fills the gap between lofty concepts of how AL should be defined theoretically and what AL is conceived of in practice by teachers. Our representation of AL is consistent with contemporary concepts of AL, but it adds specificity and practical relevance. It enhances the opportunity to sustain teachers’ AL by furtherer integrating AL as part of teaching competence. The representation of AL, in seven interrelated aspects, may guide teachers’ development of AL in practice because the aspects encompass the dynamic, contextual, and social situations of teachers’ assessment practice. The seven aspects of AL can be used by organisations to design professional development activities, beyond knowledge and skills, to foster teachers’ AL and to embed these activities within teachers’ assessment practices. Further research is needed to strengthen the understanding of AL as a dynamic, contextual and social process and to explore how teachers develop AL over time.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Kitty Meijer

Kitty Meijer is a PhD student at Open University of the Netherlands. Her research focuses on teachers’ assessment literacy within the field of higher professional education. She also works as a teacher at HU University of Applied Sciences Utrecht and is affiliated with the Research Group Vocational Education.

Liesbeth Baartman

Liesbeth Baartman is a specialist in research on assessment and evaluation in higher professional and vocational education. She holds a professorship at the HU University of Applied Sciences Utrecht and is affiliated with the Research Group Vocational Education and the Teaching & Learning Network for university-wide learning and development.

Marjan Vermeulen

Marjan Vermeulen is a specialist in research on professional learning and development of teachers with a special interest in human resource development for teachers. She holds a professorship on teacher professionalisation at the Open University of the Netherlands and has a special professorship regarding collective learning between organizations funded by the University of applied sciences de Kempel and the education consultancy organisation OMJS.

Elly de Bruijn

Elly de Bruijn is a specialist in research on teaching and learning processes in the context of vocational education. She holds a professorship on Vocational Education Research at the HU University of Applied Sciences Utrecht and is also honorary professor of vocational education at the Open University of the Netherlands.