Teaching and learning about the Holocaust: a systematic scoping review of existing research

ABSTRACT

This scoping review aims to map and summarize practice-based research that evaluates outcomes in teaching and learning about the Holocaust (TLH). It focuses on providing an overview of the characteristics of existing literature and synthesizing the educational interventions’ effects evaluated in previous studies. Nineteen bibliographical databases were searched for relevant empirical research published from 1945 to 2020 worldwide. After screening and relevance assessment, 117 research publications were identified for inclusion in the review. The included material was analysed by bibliometric analysis, thematic analysis of educational approaches, and narrative synthesis of outcomes. Based on the bibliometric analysis, the findings suggest that practice-based research on TLH’s educational outcomes is a neglected subfield within the academic field of TLH, with a lack of cumulative development across studies. The scoping review also showed a dominance of studies conducted with qualitative study designs. The narrative synthesis of outcomes showed that these studies provide important insights into how interventions within TLH impact students; nonetheless, these studies need to be supplemented with knowledge from quantitative studies. Without such knowledge, we have a knowledge gap that could imply that TLH interventions are implemented arbitrarily, that students do not learn the intended lessons, or that the lessons are not lasting.

KEYWORDS:

• Education
• pedagogy
• teaching and learning about the Holocaust
• Holocaust education
• scoping review

Introduction

Since the turn of the millennium, we have witnessed increased scholarly attention on Holocaust education, resulting in an independent international field of educational research – teaching and learning about the Holocaust (TLH) (International Holocaust Remembrance Alliance [IHRA] 2017). Teaching within TLH includes numerous educational aims beyond merely teaching about a historical event (Flennegård and Mattsson 2021). The Holocaust, as a historical event, is increasingly recognized as a vehicle for learning about social justice, human rights, and democracy (Lindquist 2011). While the relationship between learning about the Holocaust and the development of positive values, on a policy level, may seem intuitive common sense, there is a complex level of development and understanding involved in these processes (Cowan and Maitles 2005; Flennegård and Mattsson 2021). Most Western countries have implemented curricula that often include learning about the Holocaust through a lens of citizenship and moral education (Bromley and Russell 2010). Therefore, these efforts aim to facilitate learning contemporary lessons from the Holocaust (contemporary genocides and human rights issues) and learning about it (Cowan and Maitles 2011).

There are continuous discussions on how TLH should be realized in scholarly literature. These discussions concern how to best achieve the desired educational outcomes of TLH regarding different aims, content, and learning strategies. Stevick and Michaels (2013) distinguish between the perceived importance of TLH and the implications of these educational efforts in terms of effectiveness. TLH includes diverse interventions stemming from nationally implemented curricula, extracurricular interventions, as well as informal educational interventions in museums, field trips, and educational efforts connected to Holocaust Memorial days. This variety of approaches makes it difficult to define the nature of TLH (Cowan and Maitles 2005). Gundare and Batelaan (2003) argue that TLH ‘is not, and should not be, the same everywhere’; education within TLH should be adapted to the country in question and, more specifically, depending on its history of antisemitism and involvement in World War II (Gundare and Batelaan 2003). While TLH includes human rights education focusing on anti-racism, scholars are concerned about a lack of strategies within TLH to teach antisemitism and, more specifically, contemporary antisemitism (Allwork 2019; Cowan and Maitles 2007; Foster et al. 2015).

Research within the field of TLH draws upon various theoretical and disciplinary fields (IHRA 2017), and scholars have compiled and synthesized studies to get an overview of the field of TLH (e.g. Davis and Rubinstein-Avila 2013; Gray 2014; IHRA 2017; Schweber 2011). In a full-length book, Gray (2014) examines contemporary research on Holocaust education. He concludes that there is a lack of rigorous studies conducted with robust methodologies and highly questions the quality of many studies within the field. In an empirical research review, Schweber (2011) posits that while empirical research does not resolve normative questions discussed within the field, it can ground such discussions in concrete challenges in classrooms and other learning sites. Moreover, she acknowledges a need for further research into neglected areas of TLH, such as higher education, field trips, and memorial sites. While Schweber (2011) limited her review to English-language research, the International Holocaust Remembrance Alliance (IHRA) (2017) conducted an international research mapping within TLH. This systematic review collated empirical research within TLH to ‘reveal what has been established by empirical research about the current state of education concerning the Holocaust’ (2017, 18). These systematic reviews, especially the most recent review from IHRA (2017), enhance our understanding of the characteristics of TLH. However, the rapid growth of TLH (IHRA 2017) calls for an updated overview of TLH research. Therefore, considering the uncertainty regarding the outcomes of interventions related to TLH, a review of studies examining such outcomes is highly needed. This study addresses these gaps by providing an overview of existing practice-based research related to TLH and synthesizing the educational interventions’ effects evaluated in previous studies. By doing this, the study aims to understand the workings of this subfield and its outcomes.

The present review

This paper presents results from a systematic scoping review of empirical research evaluations of outcomes within TLH. The methods used in a scoping review aim to provide a broad overview of a particular research field. While a scoping review is useful in providing a map of the research within a field, it can also help identify gaps in knowledge where researchers can make a conscious effort (Arksey and O’Malley 2005). This study aimed to map existing research that evaluated interventions within the field of TLH and was guided by the overall question: What kind of research involving educational outcome analysis has been conducted on TLH, and what kind of knowledge about success factors can this research show? More specifically, the scoping review tried to answer the following research questions:

• What kind of research exists within this field? What type and quantity of studies have been conducted?

• What kinds of interventions have been evaluated in these studies?

• Are there important nodes of research, researchers, and/or interventions?

• What tendencies in effects could be traced in the synthesized research material?

• How is antisemitism approached when teaching about the Holocaust in the interventions studied?

Methods

Systematic reviews follow formulated questions and use systematic and reproducible methods to identify, select, and appraise all research relevant to the questions under study. Two methodological principles are essential for maintaining the systematics of the review: (1) The structured and comprehensive searches for literature, (2) The strict criteria for inclusion and exclusion of studies that guide the inclusion of studies for the review, reducing the risk of the selection of studies being subject to personal biases or ‘cherry picking.’ A systematic scoping review method – which uses Arksey and O’Malley’s (2005) suggested framework for scoping reviews – was used to address the aim of this study. The overall procedure for this scoping review followed the steps outlined in the framework for scoping reviews:

1. Develop the research questions

2. Search the literature in bibliographical databases

3. Perform screening and relevance assessments according to pre-defined criteria for the inclusion and exclusion of studies

4. Categorize and analyse.

The methods for each of these steps are presented in the sections below.

Literature searches

The structured literature searches were conducted within a larger research project focused on mapping educational initiatives to prevent antisemitism (authors’ own, 2021). While the search terms were primarily chosen to find literature about educational interventions to prevent antisemitism, search terms for Holocaust education/TLH were also included for this scoping review. Comprehensive and structured searches for relevant literature were performed in 19 international bibliographic databases: CINAHL, Education Research Complete, MEDLINE, ASSIA, Criminal Justice Database, ProQuest Dissertations & Theses Global, Education database, ERIC, IBSS, PAIS, Political Science Database, PsycARTICLES, PsycINFO, Social Science Database, Soc Serv Abstr, Sociological Abstracts, Worldwide Political Science Abstracts and Web of Science. Search terms included ‘Holocaust,’ ‘Holocaust education,’ ‘Shoa,’ ‘intervention,’ ‘prejudice,’ ‘antisemitism,’ ‘education,’ and ‘school.’ Bibliographic records imported from the databases were stored in Endnote®. The literature searches were conducted by an information specialist in January 2020 (for a detailed description of the searches, please contact the corresponding author). These automated searches are the strength, but also the weakness of the method: search terms enable and limit what references are discovered and not, what records are screened and assessed in the subsequent steps.

Inclusion and exclusion criteria

In systematic scoping reviews, it is essential to formulate well-focused research questions. Population/Problem, Intervention, Comparison, and Outcome (PICO), a well-established tool used to conduct systematic reviews (Schardt et al. 2007), helped us develop and focus the research questions and clarify the inclusion and exclusion criteria (Table 1). The PICO framework defines specific populations or problems, interventions, comparisons, and outcomes that will be the focus of the review. We also added the inclusion and exclusion criteria regarding study design, language, and publication year.

Table 1. PICO framework.

Population	All kinds of populations were included, e.g. pupils in all grades, teachers and the public. We excluded studies that targeted Jewish populations, since the main purpose was to learn about how education can or may prevent antisemtism.
Intervention	Educational interventions that included components of TLH not interventions that targeted the Israel-Palestine conflict
Comparison	There was no requirement that the studies must include a comparison with alternative interventions or control groups for inclusion in the scoping review.
Outcome	There were no restrictions concerning specific outcomes, i.e. all outcomes of TLH that were evaluated in the studies were of interest. There were studies, specifically within educational research and arts research, that had a component of TLH, but where the purpose of the studies was to investigate specific research questions that were not related to TLH, these were excluded.
Study design	Practice-based empirical research using quantitative, qualitative or mixed methods design
Language	No restrictions on language
Year	Not studies published before 1945

Screening and relevance assessment

One reviewer first screened the titles of the identified records; duplicates and records irrelevant to the study were excluded. The abstracts of the remaining records were screened by two reviewers independently. The remaining records were retrieved in full text and assessed. The reviewers discussed and resolved any differences of opinion regarding the inclusion or exclusion of studies (e.g. what should count as practice-based research). Some of these borderline studies were resolved by identifying misconceptions or incorrect readings made by one of the reviewers. Another subset necessitated negotiations, leading to a more nuanced comprehension of the inclusion/exclusion criteria. Consequently, there was an iterative process of refining these criteria to respond to challenging borderline cases. The relevance assessment was done using Rayyan, a web-based tool designed for screening (Ouzzani et al. 2016).

Categorization and analysis

The first step in the mapping was to extract relevant data from the included studies. The initial data extracted were author, year, country, title, publication type, study design, intervention type, description of the intervention, population, and outcomes. This information was used to get a descriptive overview of the included studies. After this initial step, the included material was examined using three different analytical approaches to answer the research questions: (1) Bibliometric analysis of the research field, (2) Thematic categorization of learning theories used in the included studies, (3) Narrative synthesis of research outcomes.

Bibliometric analysis

Bibliographic data from the included material was analysed based on authorship and keywords. These visualizations were then used to analyse the bibliometric aspects of the research based on metadata. Such bibliometric analysis aims to identify patterns within the covered research based on the content and shared reference patterns between documents. The text-based analysis of keywords used those registered at the article level by the publisher, often chosen by the authors themselves but sometimes chosen from a list of pre-determined keywords. The algorithm considers pairwise relationships between all keywords identified in the articles, citing the institution’s publications based on how often the terms occur together in the ‘author generated’ keyword list. For the authorship analysis, a bibliographic analysis of individual researchers’ co-authorship was used to identify collaboration between the researchers. Since this data is aggregated at the author level, the clustering is often more distinct. Node size is based on the total of shares of articles for each author, meaning that authorship of a paper with four other authors yields a share of 0.20 for the author. Self-citation data is not removed since it is not an issue for informational purposes. The records were exported in RIS format, and the software tool VOSviewer (Van Eck and Waltman 2014) was used to analyse the data and present the findings in visualizations.

Thematic analysis of educational approaches

One way of discerning possible connections between input and outcome within systematic reviewing is to analyse the program theories behind the interventions (Pawson 2006). Program theory is not necessarily a term frequently used within educational research. However, it is a key term in evaluation studies on how interventions of various kinds may bring about the desired result (Funnell and Rogers 2011). A program theory is often defined as a theory or a model of how an intervention should operate to gain desired outcomes. Drawing on systematic reviews that use analyses of program theory, we suggest that the analysis of pedagogical approaches embedded in different educational programs, methods, and curricula is a path to discerning possible connections between the input and outcome. Pedagogical approaches include ideas regarding the factors that drive and lead to change, spread knowledge, and facilitate a better understanding among students. Even though education facilitates learning, the latter is a multifaceted process.

Focusing on learning theories and the notion of a theory of change, we have read and coded all publications extensively and repeatedly in search of these critical mechanisms and the pedagogical theories implicitly or explicitly assumed in the interventions studied. The initial categorization was conducted by two educational researchers who independently analysed the included studies. The categories were inductively identified but informed by the two educational researchers’ previous knowledge of established learning theories. The process was then cross-checked by a blinded reviewer who identified inconsistencies. The inconsistencies and potential rationales were discussed, and the categorization changed accordingly. These discussions clarified inconsistencies due to interpretation, whereas the categories were elaborated on to help distinguish between how we defined the learning theories and the most important mechanisms. In brief, the five categories were defined by these terms: (1) self-reflection, (2) metacognition, (3) learning about the Holocaust, (4) constructivism/pragmatism, and (5) aesthetic and emotive learning. A sixth category was added for studies where we could not discern pedagogical intentions, theories of change, or the actual use of mechanisms beyond intervening.

Narrative synthesis of research outcomes

The outcomes from the included research material were compiled by narrative synthesis (Dixon-Woods et al. 2005; Snilstveit, Oliver, and Vojtkova 2012). The narrative synthesis aimed to examine if it was possible to draw any conclusions about the effectiveness of the educational interventions evaluated in the included research material. The first step in the narrative synthesis was to extract information about the types of interventions evaluated in the studies and their results. One researcher manually gathered information about the studies’ findings and summarized them in Microsoft® Excel charts. The synthesis of the outcomes was structured according to the five (+one) categories of learning theories found in the thematic categorization of learning theories. Within each category, the studies were placed in subcategories according to which intervention type or type of outcome they had evaluated. The summary of the state of knowledge for each category was based on a qualitative analytical approach where four main parameters were considered: (1) the study design and overall quality of the included studies; (2) if the effects of the interventions reported in the studies were consistent or if there were important inconsistencies across studies; (3) the characteristics of the evaluated interventions; and (4) the effect sizes. These parameters were then used to conclude any tendencies in the effects of the interventions. The narrative synthesis aimed to give an overview of the characteristics of the existing research and tendencies of effects to provide guidance for future research. This meant that the quality of individual studies was not assessed in-depth, nor were questions, such as if specific types of research with certain theoretical foundations or settings were more valid than others.

Results

The structured database searches resulted in 21,126 unique publications after duplicates were removed. The 21,126 publications were screened by title and abstract for relevance by one reviewer, resulting in 1892 articles. Two reviewers read the abstracts of these articles. A total of 161 publications were then read in full text by two reviewers, independent of each other’s judgments, and 117 were included in the scoping review of TLH. A detailed illustration of the literature search and relevance assessment process is provided in the flow chart in Figure 1.

Figure 1. Screening and relevance assessment phase.

Overview of the included studies

Of the included literature, 65 were journal articles, 35 were doctoral theses, 10 were book chapters, six were research reports, and one was a book. The majority of the included literature was published within the last 20 years. Figure 2 shows a growth in the number of publications that evaluates TLH in recent years. An overview of the studies’ characteristics is provided as supplementary material (Appendix A, available online).

Figure 2. Number of publications presented by year of publication.

Although the research was conducted in different countries (Table 2), most of it was conducted in the USA (n = 77), followed by the UK (n = 13). No research was conducted in the Nordic countries, i.e. Denmark, Sweden, Norway, Finland or Iceland.

Table 2. Included studies sorted by country.

Included studies sorted by country (n)
USA	77
UK	13
Israel	1
Scotland	7
Canada	3
Netherlands	3
Poland	2
Germany	7
UK/USA	1
Latvia	1
South Africa	1

The studies used various designs (Figure 3). Most studies used qualitative methods (57%), and the most common study design was case studies using interviews and classroom observations. Among the quantitative studies (29%), four randomized controlled trials, ten quasi-experimental studies, and other surveys were used to evaluate interventions. Data in mixed-method studies were commonly collected through interviews, documents, and surveys.

Figure 3. Percentage of included studies presented by study design.

Bibliometric analysis of included studies

Within the practice-based TLH research mapped by this review, most researchers seem only to publish once. Figure 4 shows the number of first-author publication researchers within the included material. The figure thus visualizes the odd phenomenon that 92 people have been the first author of only one article. They may have been co-author of other papers, but they occur only once (in the material) as the first author. Similarly, five people occur as the first author twice, two as the first author of three articles, and finally, two more have first-authored four papers. Visualizing the included material can identify active researchers critical to developing knowledge within this research area. Since it was only possible to include first-author publications in this kind of material description, we also mapped co-authorship of the publications. This makes it possible to identify essential nodes of researchers and research environments and therefore functions as a complement to the first author publication analysis.

Figure 4. Number of publications per first author.

Among the 117 publications in the scoping review, few researchers published more than one paper within the specific subfield of practice-based TLH (see Figure 5). Several authors published a single article or doctoral dissertation without continuing research within this area. Fifteen doctoral dissertations in educational science were included, where the doctoral student does not seem to have continued their research. Most of those with a research career are active in areas other than TLH. Several are or have been active in other parts of educational science (n = 17). Many researchers have also been active in social justice, tolerance, and multicultural education (n = 10). A few researchers in history, sociology, anthropology, psychology, and Jewish studies had a single publication in this field (n = 8).

Figure 5. Co-authorship map.

The bibliometric illustration below (Figure 6) shows which keywords are used in the 117 studies, how they refer to each other, and how their frequency has developed over time. Unsurprisingly, ‘Holocaust’ is a common keyword, hence the node size. The keyword is strongly associated with ‘education,’ ‘history,’ ‘empathy,’ and ‘teachers’ but very weakly associated with ‘antisemitism,’ which is also a minimal node, that is, rare among the keywords used to represent the content of studies. In contrast, the keyword ‘Jews’ is common, strongly associated with ‘death’ but not at all associated with antisemitism. In 32 of the 117 TLH studies, antisemitism is not mentioned at all, and in 27, it is only mentioned in passing or treated rather briefly. In 43% of the studies, antisemitism is neither presented nor a part of the analysis.

Figure 6. Map of keywords used in the included studies.

Narrative synthesis of outcomes

This section provides the findings from a narrative synthesis of outcomes. The narrative synthesis is structured around the five (+one) categories of learning theories presented earlier in the method section. The narrative synthesis aimed to summarize the findings of the included studies regarding different outcomes related to student learning to explore if there were recurring tendencies in the effects of the educational interventions. Some of the studies could have been classified into more than one category, but this was only true for quite a few; hence, it was decided to single them into one of the five categories they belonged to. If they did not belong to any of the five categories, they were sorted into the uncategorized section. A complete reference list of the included studies is provided as supplementary material (Appendix B, available online).

Self-reflection

The first category comprises programs intending to sustain or create a willingness and capacity to develop self-reflection among the students. The teaching process aims to guide the students to become self-aware by engaging in the program’s content and discussions with their peers, teachers, and significant others. This approach can be recognized in most teaching, but specific to this category is the intention that students should learn not only about the Holocaust but also from the Holocaust.

Twenty-five studies were included in this category. Among these publications, 18 studies were conducted with a qualitative study design, five with a quantitative study design, and two with mixed methods. The studies were conducted in the USA (n = 19), Canada (n = 3), Poland (n = 1), Germany (n = 1), and South Africa (n = 1).

A common component in the interventions in this category is witness testimonies. Nine studies evaluated the impact of interventions that include this component. The results of qualitative studies evaluating iWitness technology indicated that students found witness testimonies memorable, meaningful, and robust (Cole et al. 2015), that iWitness promoted students’ development of empathy and contributed to the humanization of the Holocaust. Other kinds of witness testimonies were explored in five studies. These qualitative studies indicated that students better understood that the survivors are not just symbols of the Holocaust but also individuals living today; this increased students’ empathy and openness to diversity and nuanced their knowledge and attitudes towards the Holocaust.

Four studies evaluated different types of general Holocaust education courses. The results of the studies showed that preservice teacher students cultivated a deeper understanding of diversity, social justice, and their own beliefs and biases, and that a professional development program strengthened teachers’ relationships with their students.

Twelve studies focused on the pedagogical program or model named ‘Facing History and Ourselves’ (FHAO). It is a long-lasting, well-spread, and well-known program within the TLH field. The studies show that this standardized program can be used in various ways. Three studies evaluated its comparative effectiveness. The findings of these studies indicated an increase in moral reasoning and knowledge about the Holocaust. These findings correspond to the results of qualitative evaluations of FHAO. In one study, there were no statistically significant changes in self-esteem, internal/external locus of control, or acceptance of self and others compared to the control group, indicating that the changes in the FHAO group could be due to chance.

Summary of the state of knowledge

There is a lack of robust studies evaluating the effectiveness of interventions that aim to foster self-reflection and encourage students to learn not just about the Holocaust but also from the Holocaust. Many qualitative studies give rich and detailed descriptions of educational interventions. These studies provide teachers and other stakeholders with insights into the workings of these interventions. However, they offer little information about the effectiveness of the interventions on a more general level; information that is a crucial complement to these qualitative analyses. Witness testimonies seem to be highly valued and identified as effective in the qualitative analyses by students. These tendencies of effectiveness merit consideration, but there is a need for studies of larger populations that evaluate the comparative effectiveness of the intervention. The qualitative studies on FHAO indicate positive effects, confirmed by the quantitative studies on educational outcomes related to moral reasoning and content knowledge. However, since these studies are rather old and the results of one important study showed no significant effects on several outcomes, there is a need for more studies to further to draw any firm conclusions about effectiveness.

Metacognition

The second category comprises programs that encourage students to develop their metacognitive abilities, use and challenge their intellectual capacities, and reflect on others’ thoughts and worldviews to develop their ability to make connections and draw conclusions. Metacognitive learning is associated with self-regulation and self-regulated learning (Dinsmore, Alexander, and Loughlin 2008), where active engagement with an object affects the subject. In contrast to the self-reflection category, students are expected to reach these conclusions and develop these abilities without being deliberately challenged in the educational context. Instead, students identify the cognitive goal of their learning and think about how to achieve it. This emphasizes higher-order thinking skills, whereas teaching is based on basic skills; consequently, students’ abilities to recognize what knowledge they lack and how to develop and translate it into action are not sustained (Kaplan 2008). This category included 22 publications – ten studies were conducted with a qualitative study design, eight with a quantitative study design, and four with mixed methods. They were conducted in the USA (n = 13), the UK (n = 2), Scotland (n = 1), Latvia (n = 1), the Netherlands (n = 1), Poland/USA (n = 2), and Poland/Scotland (n = 2). In the following section, the findings from these studies are discussed based on three main types of intervention: (1) study visits, (2) TLH in the classroom, and (3) university courses.

Qualitative studies of study visits to Holocaust memorial sites found that the participants (both students and teachers) experienced the interventions as valuable and deepened their understanding of the Holocaust. Elmore found that visiting such sites not only increased students’ knowledge about the Holocaust but also increased their tolerance of diversity after the intervention compared to before; this effect also persisted four months after the visit. This finding corresponds to the findings in the qualitative studies. Contrastingly, Short indicated that the study visit alone did not impact most students’ ability to link events related to the Holocaust to contemporary human rights issues. This finding indicates that students might need further educational support to make these connections.

Four studies explored the impact of university courses on students’ awareness of prejudice and tolerance. Qualitative studies indicated an increase in students’ motivations to combat prejudice. One study concluded that only a small number of students became aware of prejudice within themselves. Wolpow et al. evaluated a university course for teachers in a quasi-experimental study. The participants in the group that took the university course had higher self-efficacy than those that did not.

Nine studies explored the impact of TLH in classrooms on students’ ability to make connections to contemporary issues related to human rights. These studies indicated an increase in students’ perceived knowledge about human rights, but a lack of understanding of the meanings of antisemitism and genocide (Cowan and Maitles 2005). They showed improved student teachers’ content and pedagogical knowledge and increased students’ awareness of racism targeting Jews. Two experimental studies showed no statistically significant change in prejudice and affinities regarding the diversity between the intervention and control groups.

Summary of the state of knowledge

There is a lack of robust educational outcome metrics and evaluation designs for this intervention category influenced by metacognitive learning strategies. Both the interventions and study designs are highly heterogeneous, making it difficult to conclude effectiveness on a more general level. While the qualitative studies included in this material give critical insights into the specificity of the interventions, they only give anecdotal evidence on the more general effects of these kinds of interventions and the impact on students in the longer run. The quantitative evaluations showed inconclusive effects due to inconsistency between studies.

Learning about the Holocaust

This category identifies program studies that foreground the content-, factual-, and historical knowledge about the Holocaust. The interventions often target basic thinking skills, such as remembering names and important years and events. This is often considered traditional teaching because it is straightforward, and its question, answer, response (QAR) structure focuses on gaining knowledge of the content without any particular assumptions about individual growth or insights. Eight studies were included in this category. Three studies were conducted with a qualitative study design, three with a quantitative study design, and two with mixed methods. The studies were conducted in the USA (n = 3), UK (n = 1), Poland (n = 1), Poland and USA (n = 1), and Germany (n = 2).

Six studies evaluated learning outcomes related to historical knowledge of the Holocaust. The results of qualitative studies indicated that field trips as a component of TLH were perceived as meaningful and insightful by participants. These positive findings were confirmed in a randomized controlled trial [RCT] that found statistically significant effects on students’ historical knowledge about the Holocaust.

Two studies evaluated outcomes related to antisemitism and general political tolerance; these quantitative studies did not significantly affect those outcomes. In a quantitative study, a course devoted to the study of genocide and the Holocaust did not show statistically significant effects on levels of antisemitism or general political tolerance. Another study showed that only 16% of grade 7 students knew what antisemitism meant, and 26.7% knew what genocide was. This compared to 44.9% who correctly identified the meaning of Islamophobia, some three-quarters of students who knew what homophobia referred to, and 90.7% who knew what racism was after learning about the Holocaust in primary school.

Summary of the state of knowledge

The diversity of interventions and study designs makes it hard to synthesize the results and draw conclusions about the effectiveness of these interventions on a general level. While some studies that evaluated different types of field trips showed promising findings, other studies evaluating in-classroom interventions reported no significant effects on a general level.

Constructivism or pragmatism

Twenty-five studies have focused on teaching inspired and influenced by constructivism or pragmatism based on notions of an intrinsic urge to learn that can be challenged and stimulated by making teaching meaningful to the students. There were 13 studies conducted with a qualitative study design, eight with a quantitative study design, and four with mixed methods. The studies were conducted in the USA (n = 17), UK (n = 4), Scotland (n = 2), and the Netherlands (n = 1). These could roughly be divided into educational interventions: study visits, multimedia interventions, and teaching in the classroom. These interventions aim to harness ‘learning by doing’ or encourage students to make inquiries or interact with materials or peers to construct their knowledge. Theories of change based on social constructs are often related to ground-breaking theories developed by the poster children of pedagogy: John Dewey’s pragmatism and Lev Vygotsky’s sociocultural theory. These influential theories are based on how knowledge can be adapted for pragmatic use and learning as a process dependent on culture, context, interactivity, and the support of competent others who can guide learners or help them scaffold their learning (Säljö 2014). These practices are very common points of departure for teaching in the Western world and unsurprisingly, some models are based on these kinds of collective, social learning theories, often based on problematisation and challenge.

Study visits are evaluated in eight publications – all reported similar findings indicating that the experiential components in these interventions were valued by the participants and raised awareness and nurtured reflection. Some studies indicated that the interventions also led to increased emotional empathy.

Four publications studied the impact of different multimedia interventions. The results from both the qualitative and the quantitative studies indicate positive effects on students’ experience, educational outcomes, and changes in emotional empathy and civic engagement.

Thirteen publications evaluated classroom interventions. Qualitative studies indicate that teaching in the classroom influenced by constructivism or pragmatism seemed to evoke reflection and enhance students’ critical thinking. The qualitative studies also suggest that facilitating learning about the Holocaust by promoting students’ internal urges through interaction and reflection was a successful teaching method. The results of the quantitative studies correspond to those findings and show positive effects on students’ knowledge about the Holocaust (Cowan and Maitles 2007) and antisemitism (Cowan and Maitles 2007), as well as increased levels of reasoning about social and moral issues. While indicating positive effects on students’ knowledge of antisemitism, one study indicated that this did not affect students’ attitudes toward Jews (which deteriorated after the intervention).

Summary of the state of knowledge

Only a few small studies evaluate the effectiveness of interventions based on social theory/pragmatism concerning educational outcomes related to knowledge, social lessons, personal development, and empathy. However, in the included material, some tendencies in effects are visible. The findings of the studies indicate that TLH influenced by social theory or pragmatism foster a dynamic learning experience among students and enhance their ability to reflect and increase their emotional empathy. The different experiential learning experiences seemed highly valued among students; there are some indications that the effects might not be sustained in the longer term. These are only indications and tendencies, and the character of the included research made it difficult to draw any firm conclusions about effectiveness. Instead, more research that evaluates effects on larger populations and with longer follow-up times is needed to draw more robust conclusions.

Aesthetic and emotive learning

Sixteen studies have applied art and drama to teaching and learning about the Holocaust. Five studies were conducted with a qualitative study design, five with a quantitative study design, and six with mixed methods. The studies were conducted in the USA (n = 10), UK (n = 4), and Germany (n = 2). Art and drama are frequently interwoven in programs in the other categories as well, often for multisensory learning. Within this specific category, art, and drama engage the students with the topic of the Holocaust. It is not so much about making the lessons from the Holocaust stick, nor is it about learning more about the Holocaust but mainly about getting more involved emotionally. Occasionally emotions are at the core of the intervention, and emotive responses to TLH are researched.

Eight studies evaluated the effects of art and aesthetic interventions on social lessons, tolerance, and the moral implications of the Holocaust. The tendencies in the effects reported in these studies are coherent across qualitative and quantitative studies. While there is significant heterogeneity in the gathered material concerning educational outcome metrics, as well as interventions and evaluation methods, the qualitative studies showed similar findings: that the students get involved at an emotional level through the art and aesthetic interventions and that this seems to contribute to students learning about the social and moral lessons of the Holocaust (Gray 2014). The results from two RCTs show a statistically significant effect on educational outcomes related to social and moral lessons compared to the control groups.

Three studies evaluated the effectiveness of integrating different art interventions in teaching the Holocaust to students. The tendencies in effects visible in this small number of studies indicate a somewhat uncertain effect of art on factual knowledge about the Holocaust. Two quantitative studies reported results indicating that the art interventions did not significantly impact students’ factual knowledge about the Holocaust. A qualitative study supported this, but this study also suggested that art can function as a way to increase students’ interest in learning.

Five studies explored student learning through art and aesthetics. In these studies, the Holocaust was primarily used as a tool to explore student learning. The studies are very heterogeneous with regard to intervention, study approach, and purpose, which makes it hard to draw any general conclusion about tendencies in effects.

Summary of the state of knowledge

Few and only small studies have evaluated the effectiveness of art and aesthetics interventions on educational outcomes related to knowledge and the social lessons of the Holocaust. However, in the included material, some tendencies in effects are visible. While the findings of the studies indicate a very uncertain effect of art and aesthetics on students’ factual knowledge about the Holocaust, they do indicate that these kinds of interventions could increase students’ learning about the social and moral lessons about the Holocaust. It has to be acknowledged that these are only indications and tendencies and that the character of the studies made it hard to draw any firm conclusions about effectiveness. Instead, more research is needed to be able to draw more robust conclusions.

Uncategorized

Twenty publications evaluated programs we could not categorize due to lack of information, incomprehensibility, and the absence of a theory behind the program. There were 15 studies conducted with a qualitative study design, one with a quantitative study design, and four with mixed methods. The studies were conducted in the USA (n = 11), the UK (n = 3), Israel (n = 1), Germany (n = 1), Scotland (n = 2), Netherlands (n = 1), and USA/UK (n = 1). Due to the heterogeneity of the material in this category, it was not appropriate to synthesize the findings. Nonetheless, the description of the studies provided in the supplementary material (Appendix A, available online) could be used to inspire further research.

Discussion and conclusions

The overall purpose of this review has been to contribute to an increased understanding of the subfield of practice-based research on outcomes of TLH interventions and the educational outcomes of this research. The overall question guiding the scoping review has been: What kind of research involving educational outcome analysis has been conducted on TLH, and what kind of knowledge about success factors can this research show?

The bibliometric analyses showed that the studies identified did not build or develop on previous research (with a few exceptions, see Figure 5). The findings indicated almost no cumulative development across studies, at least not if development means using and refining previous findings. Most researchers publish only one study in the field. Consequently, while scholars have emphasized a growing research field of TLH (IHRA 2017), the results of this study imply that practice-based research on educational outcomes of TLH is a neglected subfield within the TLH. This conclusion corresponds to findings from a previous literature review (Gray 2014).

Another research question was how antisemitism is approached in the gathered material. The bibliometric analysis of keywords showed that TLH educational initiatives were only weakly related to antisemitism. In 43% of the included TLH studies, antisemitism was not mentioned at all or was only mentioned in passing or briefly addressed. Further, in evaluating educational outcomes, it is uncommon to evaluate outcomes that relate to antisemitism. Instead, the findings of this scoping review indicate that TLH initiatives seem to be more about human rights in general or racism and homophobia than about antisemitism. Furthermore, this pattern could indicate an expectation that the fate of the Jews during the Holocaust is not only tied to Jews but to other groups as well. If so, the fate of the Jews will be used as an educational resource to arouse empathy concerning how other groups are suffering today, but rarely to counter or prevent antisemitism. The invisibility of the prominent role of antisemitism leading to the Holocaust hindered the potential of TLH in preventing antisemitism and could, in the worst-case scenario, contribute to antisemitism.

The narrative synthesis of outcomes in the scoping review aimed to examine the tendencies in the effects of different educational initiatives. The scoping review showed a dominance of studies conducted with qualitative study designs in this subfield of TLH. This is obvious since qualitative studies constitute a solid foundation and an established tradition in education science. As shown in the outcome evaluation, a lack of robust quantitative studies evaluating long-term effects and effects on larger populations makes it difficult to draw any firm conclusion about these effects. In this kind of educational outcome evaluation, findings from qualitative studies provide important indications of how the interventions impact students, but these findings need to be supplemented with knowledge from quantitative studies that evaluate the comparative effectiveness of the interventions. Without such knowledge, we have a critical knowledge gap, implying that TLH interventions are implemented arbitrarily and that students do not learn the intended lessons we want them to learn or if the lessons they learn are lasting.

Even if there are no robust findings concerning educational outcomes, such as any hard evidence for how to best carry out TLH, there are important lessons to be learned from some studies. Educational programs or methods developed based on learning theories inspired by pragmatism or constructivism could provide knowledge about what mechanisms and educational prerequisites improve teaching. It cannot be determined if this is due to any specific learning theory since pragmatism or constructivism are the dominant foundation for teaching in the Western world in general, and not least in the USA. At a more detailed level, the reason might be related to mixed methods in teaching, which is also common practice within ‘learning by doing’ approaches, that is, pragmatism. The findings do not provide hard evidence that students taught according to these models develop more insights concerning tolerance or a sustained belief in democracy, but some results indicate positive changes in those outcomes and that they learn more and better about the Holocaust. This should be compared with teaching models that advocate that straightforward teaching about the Holocaust is sufficient. Further developments and research using mixed methods in teaching are thus encouraged.

Several institutions worldwide produce teaching materials, provide in-service training for teachers, and supply teaching models, but only a few of these institutions are found in the academic literature reviewed in this study. Therefore, the findings in this scoping review call for more research evaluations of TLH. If this had been done previously, there would already be an academic field with the capacity for cumulativity. This is of particular importance at the major institutions around the world: those visited by hundreds of thousands or millions per year, those who train teachers in vast numbers, and those who produce teaching and learning materials and develop methods and philosophies. They are all performative in shaping how TLH is conducted. Noteworthily, the TLH teaching field is already overflowing with educational instructions, political aspirations, and several international bodies that offer various guidance, recommendations, and best practices. Moreover, as noticed by Marcus et al., the educational field is transforming as new teaching technology replaces witnesses of the Holocaust in classrooms. These multiple instructions given to teachers as well as the generational shift (i.e. replacing real persons in the classroom with technology) underline the importance of continuing to develop TLH as an academic research field.

This scoping review has focused on mapping research evaluating TLH interventions, their characteristics, and tendencies of effects. Constraints of the method consist of automated searches and search terms that may exclude potentially relevant material, but also an analysis that encompasses an extensive range of materials across various contexts. Thus, it is limited to presenting a broad overview of the existing literature rather than any in-depth analyses of causal inferences or core mechanisms that could offer a fuller understanding of the implications of individual studies. The strength of the scoping review is its rigorous methods of gathering and reviewing the existing research within the field and its contributions to increasing the understanding of interventions and outcomes within TLH. In summary, this scoping review shows ‘the state of the art’ within TLH research, and hopefully, the information presented in this paper can help guide future research within this field.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Swedish Ministry of Education.

Notes on contributors

Isabella Pistone

Isabella Pistone, PhD student in theory of science at the department of philosophy, linguistics and theory of science, University of Gothenburg. Pistone has conducted several systematic reviews within different areas such as public health, education and social care. Her research interests include evidence-basing, quality improvement and care practices in welfare. She is currently working on a thesis where she explores applied forms of science- and technology studies (STS) in relation to issues concerning evidence-basing in welfare practices.

Lars M. Andersson

Lars M. Andersson, senior lecturer in history at Uppsala University. Anderson has mainly done research on Swedish antisemitism and refugee policy but has also published high school textbooks on history and religious studies, anthologies on contemporary history, counterfactual history, images as historical sources, liability issues and moral debate in historical accounts, migration and Swedish Jewish history. Furthermore, he has co-edited three Festschrifts. In 2002–2006, he was the editor of Historisk tidskrift and has served as editor of the Lagerbringbiblioteket series with Ulf Zander. Currently he is the editor of Personhistorisk tidskrift and on the editorial board of Historisk tidskrift, Scandinavian Jewish Studies, Opuscula Historica Upsaleinsia and Uppsala Jewish Studies.

Allan Lidström

Allan Lidström at the department of Philosophy, linguistics and theory of science, University of Gothenburg. Lidström is a PhD student in theory of science. His research concerns how marginalized groups mobilize experiential knowledge to engage in research and politics using disability rights organizations for persons with intellectual disability as an example.

Gustaf Nelhans

Gustaf Nelhans at Swedish School of Library and Information Science, University of Borås. Nelhans is a theorist of science and has focused his research around the performativity of scientometric indicators as well as on the theory, methodology and research policy aspects of the scholarly publication in scientific practice. He is also specialized in the theory and methods of bibliometric analysis.

Tobias Pernler

Tobias Pernler at the Biomedical Library, Gothenburg University Library, is an information specialist who is specialized in methods for structured database searches. He has a long experience in teaching and conducting systematic literature searches.

Jennie Sivenbring

Jennie Sivenbring is a senior lecturer at The Segerstedt Institute at the University of Gothenburg. She holds a Ph.D. In Child and Youth Studies and her research primarily focuses on policy on prevention against violent extremism on a Nordic level, and the management of children and youth within the area violent extremism and terrorism.

Morten Sager

Morten Sager is an associate professor in theory of science and research, and the founder and director of the master’s program in evidence-basing at the University of Gothenburg. Together with a group of several doctoral students and senior researchers at the department of philosophy, linguistics, and theory of science, he combines the production of evidence in the form of systematic reviews with epistemological analyses of evidence-based practices in health care, social work, public health and education.

Christer Mattson

Christer Mattsson is an associate professor and the director of the Segerstedt Institute at the University of Gothenburg and senior lecturer in pedagogy. His research focuses on how society acts to prevent people from joining movements, ideologies, or religious systems that use violence to achieve their goals. Furthermore, he examines the life trajectories of people who have joined, and in some cases, left white supremacy environments. He has a particular interest in how teaching and learning about the Holocaust is deployed as a pedagogical mean to foster democracy.