Interventions and evaluation of intercultural competence of students enrolled in higher education – a scoping review

ABSTRACT

Over the past decade there has been an increase in scientific publications addressing intercultural competence (IC) of students. The sheer volume of publications available makes it difficult to determine the extent, breadth, and nature of research within the area. The aim of this scoping review was to describe the state of peer reviewed research related to IC, including academic disciplines addressing the issue, regions of the world conducting research, types of interventions used to foster IC and how outcomes are being evaluated. Six databases were searched, resulting in 15,128 articles. A total of 464 met the inclusion criteria. A trend was observed towards studying IC in interdisciplinary student populations as well as a post-COVID-19 trend towards more online interventions. Most research was conducted in North America (n = 198; 42.7%) within the discipline of education (n = 87; 18.8%). The most common intervention was pedagogical approaches delivered at the students’ home institution (n = 161; 34.7%). Results highlight a gap in research from the Global South and a lack of consensus regarding appropriate tools for evaluating IC. Continued work is required to determine the effects of specific interventions and to support educators in identifying appropriate tool(s) for measuring outcomes.

KEYWORDS:

• Education
• global citizenship
• global competence
• internationalisation
• intercultural
• post-secondary

Introduction

Over the past decade, internationalisation, prompted by geopolitical and societal shifts, has evolved to become a key issue for reform within the education sector (Hong & Cheon, 2017). Preparing students for the complexity of an increasingly global world is now considered one of the most important goals of internationalisation for higher education institutions (HEIs) (Rumbley & Sandström, 2019). To achieve this goal, HEIs must work actively to provide students with a variety of experiences specifically aimed at fostering their intercultural competence (IC). Since 2007 there has been an exponential increase in IC research (Peng, Zhu, & Wu, 2020). To manage the sheer volume and breadth of publications related to IC, to explore developments and to identify knowledge gaps, it is necessary that researchers employ systematic methods to synthesise existing studies and map relevant trends.

Scoping reviews

This study presents results of a scoping review which aimed to describe the state of peer reviewed higher educational research related to IC. Scoping reviews are becoming an increasingly popular approach for synthesising research data and offer researchers the opportunity to examine the extent, range and nature of research that has been conducted on a specific topic (Pham et al., 2014; Pollock et al., 2023). Scoping reviews are most appropriate when addressing broad research questions (e.g. what types of interventions and outcomes are being reported) (Pollock et al., 2023). Their broad nature means that they are descriptive by nature (Campbell et al., 2023) and often employed as a precursor to systematic reviews which are used to address more precise, targeted questions (e.g. What is the effectiveness of intervention A?) (Pham et al., 2014). The methods for conducting a scoping review are well documented in several seminal publications, most notable being the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) (Tricco et al., 2018) and the JBI Scoping Review Methodology (Peters et al., 2020). These publications recommend an iterative approach which involves 1/conducting a systematic search of relevant databases, 2/screening studies against pre-determined inclusion criteria, 3/extraction and synthesis of relevant data and 4/interpretation and reporting of results.

Intercultural competence in the context of higher education

In an era marked by globalisation, continuous global political shifts and use of online and virtual platforms, there is an expectation, and we argue a social responsibility, for higher education institutions to provide students and staff with the competencies they need to navigate and engage with diverse cultures and societies (Gregersen-Hermans, 2017). This includes providing competencies to help them coexist effectively and peacefully with people from different cultures but also competencies to communicate and cope across a variety of cultural contexts. As universities acknowledge their role in supporting the development of intercultural competencies of students and staff we are seeing a substantial shift in their strategic approach to internationalisation, as well as in the internationalisation strategies of higher education authorities (Austrian Federal Ministry of Education, 2020; Sinjumpasak & Mhunpiew, 2022; Swedish Govenment, 2018). While student and staff mobility were previously the major strategic priority in the higher education sector, it is now recognised that a broader and more inclusive strategy is desired, supporting IC skill development across the entire university community and not limiting the opportunity to those who participate in study abroad programs (de Wit & Deca, 2020). This strategic shift is reflected in concepts such as “comprehensive internationalisation”, and “internationalisation of curriculum”, which have become increasingly common in discourse related to internationalisation of higher education over the past decade. “Internationalisation at home”, is another example representing a clear shift in the strategic priorities of higher educational institutions. “Internationalisation at home”, involving the integration of international and intercultural dimensions into domestic learning environments (Beelen & Jones, 2015), received a great deal of attention during the COVID-19 pandemic when opportunities for physical mobility dissolved and educators were forced to explore digital alternatives to support international encounters.

Expanding the scope of initiatives designed to enhance intercultural competencies poses a considerable challenge to higher education institutions. Not only must they identify and implement evidence-based IC interventions on top of already dense curricula and with limited human resources, but they must also establish effective and efficient means of evaluating outcomes. We argue that a succinct summary of IC interventions and evaluation tools will support higher education institutions in narrowing down the alternatives available to them and in helping them to identify relevant sources of evidence.

Intercultural competence – theoretical constructs

Definitions of intercultural competence have arisen from a wide variety of disciplines which has likely contributed to confusion regarding the concept and the terms used to describe it. To date there is no consensus upon a single term or theoretical explanation of intercultural competence and terms such as cross-cultural awareness, global competence, intercultural sensitivity and even intercultural communication are used interchangeably (Odag, Wallin, & De Santis, 2015). What is generally agreed upon is that IC does not come naturally and evolves through an ongoing process of growth and learning. It is also generally accepted that IC is a multidimensional construct and while conceptualisations are diverse, many of these contain common elements including affective, cognitive and behavioural components (Chen, 2022; Zhang & Zhou, 2019). The affective component suggests that development of IC requires a willingness to learn and appreciate different cultures. The cognitive component requires knowledge and an understanding of different cultural norms and practices, while the behavioural component addresses the practical skills required to navigate intercultural interactions. Adaptability is a further component that is mentioned in many conceptualisations of IC but has received critique for lacking conceptual and empirical clarity (Odag, Wallin, & De Santis, 2015; Spitzberg & Changnon, 2009).

Griffith, Wolfeld, Armon, Rios, and Liu (2016) identified 26 different definitions of IC used in higher education literature and recognised that they were related to models that are common to education, training and research. These models can be grouped under 5 categories derived by Spitzberg and Changnon (2009); compositional models, co-orientational models, developmental models, adaptational models and causal models. Compositional models identify the components of IC but do not detail the relationship between these components (e.g. Deardorff, 2006; Hunter, White, & Godbey, 2006). Co-orientation models focus on the processes necessary for intercultural interactions (e.g. Fantini, 2009). Developmental models also focuses on intercultural interactions but also describes the stages through which IC develops over time (e.g. Bennett & Paige, 1993; King & Baxter Magolda, 2005). Adaptation models are concerned with how individuals adjust their communication and/or behaviour in different contexts (e.g. Berry, Kim, Power, Young, & Bujaki, 1989), and causal models are concerned with understanding various predictor variables and interactions between them (e.g. Arasaratnam, 2008). While consensus has not been reached on a single model or theoretical framework for IC we argue that the above-mentioned models can each provide a useful framework against which to conceptualise IC and that selection of a specific model should be guided by its appropriateness for the goal of the activity or research being undertaken. For example Fantini’s co-orientation model (Fantini, 1995) focuses largely on communicative mutuality and could prove useful to facilitate the study of student communication across cultures and languages. Adaptational models tend to have multiple interactants and focus on the process of mutual adjustments that need made throughout an intercultural interaction (Spitzberg & Changnon, 2009). This could be a useful model for exploring intercultural interactions and adjustments of students involved in online interactions. Compositional models tend to specify components of IC without necessarily detailing the relations among or between them. As such, they can be useful when the goal is to describe components of IC rather than understand processes (Spitzberg & Changnon, 2009).

In this review we chose to work with the Process Model of Intercultural Competence (Figure 1) (Deardorff, 2006). This model has been suggested as one of the most widely influential in higher education (Arasaratnam-Smith, 2017) and was selected as it could be used to conceptualise IC over the broad range of settings and situations that were anticipated to arise in this scoping review.

Figure 1. Process model of intercultural competence (Deardorff, 2006).

The Process Model depicts the complexity of IC as well as movement between respective components, beginning at the personal level and moving to the interpersonal/interactive level (Deardorff, 2006). The personal level comprises attitudes (respect, openness, curiosity and discovery), knowledge (cultural self-awareness, deep cultural knowledge and sociolinguistic awareness), skills (observe, listen, evaluate, analyse, interpret and relate) and internal outcomes which are observable within the individual (flexibility, adaptability, ethnorelative view and empathy). The interpersonal/interactive level addresses external outcomes which are observable through interactions with others (the effective and appropriate behaviour and communication in intercultural situations). The attitudinal components in the process model are considered most critical and reflect the starting point (Deardorff, 2006). Given that intercultural competence is a lifelong improvement process there is no endpoint.

Supporting students’ development of IC

In the context of this scoping review the term intervention represents any action taken with the specific intent of influencing students’ IC. While interventions proven to enhance IC would be of obvious interest to higher education institutions, it should be acknowledged that IC development can occur in the absence of any planned IC intervention. This could be the result of naturally occurring interactions between individuals, the campus environment, student maturation or course material that promotes students’ self-awareness by default rather than design.

Interventions aimed at fostering IC within the higher education context can be categorised within two overarching areas; pedagogical interventions or immersion experiences (Zhang & Zhou, 2019). Pedagogical interventions refer to learning materials and activities that are applied within the context of the student’s home institution. Zhang and Zhou (2019) identified four sub-types of pedagogical interventions, including culture-based teaching materials (e.g. internationally orientated course literature), classroom activities (e.g. role-playing), teaching strategies (e.g. reflective learning) and integrated intercultural programmes (civic engagement experiences). Immersion experiences are generally defined as involving a period of studying or working in a target country, this may be a short-term experience of anywhere between a week to 2 months or a long-term experience extending over many months to a year (Zhang & Zhou, 2019). It is interesting to reflect that as technology progresses this definition may well be broadened to include virtual reality based interventions (Dunn, 2019). There are contrasting opinions regarding the effectiveness of different types of interventions on IC of students within the higher education context. Zhang and Zhou (2019) suggest that study abroad initiatives elicit a greater effect on IC than other types of interventions while others warn that enhancing student mobility does not necessarily lead to increased IC (Leask, 2015). Others suggest that pedagogical activities delivered at students’ home institutions (internationalisation at home) can develop global, international and intercultural competencies as effectively or even more effectively than study abroad initiatives (Barbosa, Santos, & Prado-Meza, 2020; Soria & Troisi, 2014).

Evaluating students’ IC

Evaluation of IC has received much attention in educational literature and has been addressed using both qualitative and quantitative approaches (Deardorff, 2006). Consistent with the wide variety of definitions and models associated with IC, instruments developed to evaluate outcomes vary in the constructs and dimensions that they address (Griffith, et al. 2016). To date there is no consensus on the most appropriate instrument for measuring IC of students within the higher education sector, or even if it is possible to measure such a complex phenomenon (Borghetti, 2017; Zotzmann, 2015). In 2009, Fantini identified more than 90 instruments claiming to measure IC and suggested that the most appropriate means of assessing the concept was through a combination of quantitative and qualitative methods. In what appears to be an informal review of IC evaluation tools, Griffith, et al. (2016) identified 32 instruments used to measure IC in higher education, concluding that few meet acceptable standards of construct clarity and noting an overdependence on self-report measures. This result is supported by Müller et al. (2020) who investigated tools to evaluate social, emotional and intercultural competence of students in early secondary school contexts. From a search of almost 14 000 articles, 149 tools were identified. 90% of these were self-report surveys and questionnaires.

Study aim

To advance internationalisation efforts and research related to intercultural competence within higher education, it is necessary to understand the scope of existing studies and to identify gaps in the literature. The aim of this scoping review was to describe the state of peer reviewed higher educational research related to IC, including the academic disciplines commonly addressing the issue, the regions of the world where research is being conducted, the types of interventions being used to foster IC in students and how outcomes are being evaluated. To gain a deeper understanding of how IC is being evaluated, we also aimed to categorise components of IC included in commonly used instruments, employing Deardorff’s model of IC as a framework. Research question for the review were;

• What components of IC have been the focus of peer-reviewed research conducted to evaluate the intercultural competence of students enrolled within higher education over the past 2 decades?

• What types of interventions have been used to foster IC of students within higher education research?

• What gaps exist in the literature that should be addressed in future research?

Methods

A scoping review was conducted and reported according to the PRISMA-SCR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (Tricco et al., 2018). The study protocol was registered in the Figshare repository prior to initiating the review (https://doi.org/10.6084/m9.figshare.14159219.v1). The review protocol was initially designed to be a systematic review, however after a systematic search of the literature and recognising how vast and broad the body of literature was, a decision was made to change the method to a scoping review.

Studies were eligible to be included if they presented original research that evaluated intercultural competence of students enrolled in higher education at the bachelor and/or master level. They were required to be published in peer reviewed journals, in English or one of the Nordic languages (Swedish, Danish or Norwegian). Studies were excluded if they evaluated IC of PhD students as it was assumed that they would have developed their IC during their undergraduate/masters education and would not respond the interventions in the same manner as undergraduate and master’s students. Studies were also removed if they investigated the IC of employees, or where the primary aim of the study was to develop and validate an IC instrument. Studies were also excluded if the IC measure was subject specific (e.g. only applicable for medical students) and was not deemed by the authors to be transferrable to a broader educational setting.

A final search was conducted in April of 2023 using the following databases/platforms: APA PsycINFO (ProQuest), Education Database (ProQuest), ERIC (EBSCOhost), EBSCOhost same search of several databases, Scopus (Elsevier), Teacher Reference Center (EBSCOhost), and the Web of Science Core Collection. Detailed search strategies are presented in supplementary file 1. In an attempt to retrieve all relevant articles a broad search was constructed using terms associated with the following criteria: intercultural competence, evaluation, and students in higher education. Key terms that informed the search are shown in Table 1. The search was limited to academic publications published after 2005. This period was selected as it captured the period several years prior to the exponential increase in IC research reported by Peng, Zhu, and Wu (2020). No study design limits were imposed on the search.

Table 1. Key terms used in search strings *truncation symbol, used as a substitute for any string of zero or more characters at the end of a word.

Intercultural Competence	Evaluation	Higher education
bi-cultural bicultural cross-cultural crosscultural cultural ethno-centricity ethnocentricity ethno-relativism ethnelativism global inter-cultural intercultural inter-national international multi-cultural multicultural trans-cultural transcultural	assess* chart* checklist* evaluat* examin* index* indicat* instrument* interface* interview* inventory inventories outcome* portfolio* profile* questionnaire* rank* rate* rating* scale* self-assess* survey* test tool* valid*	advanced bachel* college* doctoral graduate* higher master* phd post-secondary postsecondary undergraduate* universit*

The search strategy was developed by a research librarian (EN) and peer reviewed by another information specialist not otherwise associated with the research project. Peer review involved proofreading the syntax, spelling and overall. A draft search strategy was developed, and additional search terms were identified with input from the project team.

Records found during the search phase were exported to reference management software (EndNote) to enable the identification and removal of duplicates. Prior to the final screening process, a calibration exercise was undertaken to pilot and refine the screening questions. Records were then screened using Rayyan, a web-based application for systematic reviews (Ouzzani, Hammady, Fedorowicz, & Elmagarmid, 2016). Selection was based on the previously described inclusion/exclusion criteria. Screening at the title/abstract level as well as subsequent screening of full-text articles was performed by the authors who were divided into teams of two. Abstracts and subsequently full text articles were first reviewed at the level of the individual and then compared within the team. Any conflicts were resolved in discussion with a third author. Individuals within each team were rotated throughout the process to avoid potential selection bias within the teams.

Data extraction

The authors individually extracted data from the included studies using an excel form that was agreed upon and tested by all authors. Data from 20% of studies was checked by a second author. Extracted data included: authors, publication year, title, journal, study aim, study design, academic level of student under investigation, discipline in which students were enrolled, world region, IC intervention, IC analysis tool/method, if validity was reported for the IC analysis tool and results. The region of world in which studies were conducted was classified according to the World Bank’s analytical groupings (World Bank, 2017). When coding interventions, study abroad programmes were divided into short-term programmes (<2 months), long-term programmes (>2 months) and those which included a combination of both short- and long-term study abroad. The two-month cut-off was selected as the European Commission’s Erasmus+ programme currently defines a long-term study abroad as more than 2 months (European Commission, 2023). In cases where specific interventions were tested the authors coded the results as 1/having a positive effect on one or more IC domains, 2/having no effect or a negative effect or 3/having an inconclusive result.

In order to further explore the extent to which studies evaluated the different elements of IC, instruments used in five or more studies were identified and items within each instrument were classified according to components with Deardorff’s Process Model of Intercultural Competence (Deardorff, 2006). Classification of IC instruments was performed by two of the authors (NR and LW) who independently classified items in each instrument and compared their responses. Any discrepancy was resolved through discussion, and a third author was available to mediate if agreement could not be reached.

To synthesise the data we followed recommendations for analysis and presentation of results in scoping reviews as detailed by Pollock et al. (2023). This was done by reporting frequencies and percentages/proportions. As scoping reviews do not address questions of effectiveness or feasibility no advanced analysis methods were applied.

Results

Figure 2 presents a flow diagram of the screening process in accordance with the PRISMA 2020 statement (Page et al., 2021). Searching the databases/platforms retrieved 22,146 records. After removal of duplicates, 15,128 records were screened at the title/abstract level. This resulted in 773 documents which were screened in full. Eight reports were not able to be retrieved in fulltext. After fulltext screening, 464 studies were included in the final review. A summary of articles is included as supplementary file 2.

Figure 2. Prisma flowchart depicting the process of identifying studies to be included in the review.

The majority of studies included in the review (56.7%; n = 263) evaluated IC of students studying at the bachelor level while 9.3% (n = 43) focused on master’s students. Some studies, 12.5% (n = 58), included both bachelor and master’s students, while the remaining studies did not specify if they included were bachelor or master’s students 21.6% (n = 100).

Most studies included students who were enrolled in a range of different disciplines (21.1%; n = 98) and the largest increase in the number of publications per year was for this group. Studies involving students from a range of disciplines had a 3-year average number of publications that was less than 1 for the periods 2005–2007 and 2008–2010 and a three-year average of 14.6 publications between 2020 and 2022. Most research involving students who were all enrolled in the same single discipline investigated students enrolled in the fields of education (18.8%; n = 87) or health care and medicine (15.7%; n = 73) (Table 2). The majority, 42.7% (n = 198), of studies were conducted on students from North America, while students from sub-Saharan Africa and South Asia were represented in less than 1% of the total studies (Table 3).

Table 2. Distribution of studies by field of study.

	Frequency	%
Education	87	18.8
Health/medicine	73	15.7
Mixed	98	21.1
Language	62	13.4
Other	38	8.2
Business/accounting	37	8.0
Not clear	35	7.5
Social work/psychology/counselling	21	4.5
Engineering	10	2.2
Liberal arts	3	0,6
Total	464	100.0

Table 3. Distribution of studies by region.

	Frequency	%
North America	198	42.7
Europe/Central Asia	97	20.9
East Asia/Pacific	83	17.9
Varied	61	13.1
Latin America/Carribean	11	2.4
Middle East/North Africa	9	1.9
South Asia	4	0.9
Sub-Saharan Africa	1	0.2
Total	464	100.0

The most common IC interventions examined (Table 4) were pedagogical interventions focusing on internationalisation of curriculum or coursework specifically developed to enhance IC (34.7%, n = 161). This was followed by study abroad programmes of varying lengths (24.8%; n = 115). Short-term study abroad initiatives were the most investigated study abroad initiative (14.7% n = 68). 19.2% (n = 89) of studies investigated IC of students enrolled in programmes without mentioning any intervention specifically tailored at fostering their IC.

Table 4. Distribution of studies by IC intervention.

	Frequency	%
Curriculum/course work	161	34.7
Study abroad programme (total)	115	24.8
(<2 months)	68	14.7
(>2 months)	40	8.6
(includes < & > 2 months)	6	1.3
(unclear duration)	1	0.2
No intervention	89	19.2
Online intercultural interactions	62	13.4
Service/community based learning	17	3.7
Peer learning/mentoring	6	1.3
VR platform	3	0,6
Other	11	2.4
Total	464	100.0

Note: Study abroad programme (total) reflects the sum of short duration programmes (<2 months), long duration programmes (>2 months) and all other study abroad variants.

The overall number of studies investigating IC showed a steady increase in the frequency of publications between 2005 and 2022, with a peak in 2021 (Figure 3). Interventions focusing on curriculum or coursework have been most popular since 2010 (Figure 4). In 2019 the number of interventions based on online web-based intercultural interactions substantially increased. Sixty per cent of the total studies investigating online intercultural interactions were published between 2019 and 2022 (n = 37/62).

Figure 3. Number of studies by year note: data from 2023 reflects publications from January and February only

Figure 4. Types of IC intervention represented in studies from 2005 to 2022

Table 4 presents studies which reported on the effects of a specific intervention on students’ IC (n = 330), 272 (82.4%) reported that the intervention has a positive effect on at least one domain, 57 (17.3%) of studies reported no effect or a negative effect of the intervention, and 1 study (0.3%) presented inconclusive results. Seventy-six per cent of interventions that were focused on internationalisation of curriculum reported a positive effect on one or more domains of IC (n = 122/161). Sixty-eight per cent of studies including a period of study abroad reported a positive effect on one or more domains (n = 79/115), and 74% of interventions based on web-based intercultural interactions reported a positive effect (n = 46/62).

Study designs used to evaluate IC are presented in Table 5. A mixed methods approach was used in 131 studies (28%) followed by before and after designs (n = 110; 24%) and cross-sectional studies (n = 100; 22%). Qualitative methods were used in 94 studies (20.2%). Apart from purely qualitative studies, all other studies used one or more self-report instruments to collect data related to IC. A total of 83 different instruments were identified, excluding those that were purpose designed for individual studies (supplementary file 3). The most commonly used instruments were the Intercultural Sensitivity Scale (Chen & Starosta, 2000) (n = 43), Intercultural Development Inventory (Hammer, 2012) (n = 39) and the Cultural Intelligence Scale/Survey (Gozzoli & Gazzaroli, 2018) (n = 23). There were 74 studies which used their own purpose-designed instrument for measuring IC.

Table 5. Distribution of studies by design.

	Frequency	%
Mixed-methods	131	28
Before & After	110	24
Cross-sectional	100	22
Qualitative	67	14.4
Case report/series	22	4.7
Case control	18	4.0
Cohort	12	2.5
Other	4	0.8
Total studies	464	100.0

In order to further understanding of the domains of IC that are commonly investigated in higher education research, items within frequently used IC instruments (reported in 5 or more studies) were classified according to Deardorff’s process model (Deardorff, 2006). Items within the Intercultural Development Inventory, used in 33 studies, were unable to be classified as this instrument is locked behind a paywall and individuals are required to attend a training course prior to accessing it. The authors were also unable to access the Multicultural Personality Questionnaire despite attempts to contact the developers. Table 6 presents a breakdown of items within the remaining IC instruments. Of the total items coded (n = 207), 114 (50%) were classified as pertaining to attitudes (e.g. respect, openness, curiosity and discovery). Less than 5% were classified as pertaining to skills, internal outcomes or external outcomes. Items that the authors could not fit within the components of the model were classified as “other” (n = 41; 20%) and included questions such as “I am developing a meaningful philosophy of life” (Global Perspective Inventory (Research Institute for Studies in Education, 2017).; “I cannot eat with chopsticks” (Inventory of Cross-Cultural Sensitivity” (Cushner, 1992) and “It is very important that a friend agrees with me on most issues” (Miville-Guzman Universality-Diversity Scale (Miville et al., 1999).;

Table 6. Number of items within IC tools that represent components within Deardorff’s process Model of intercultural competence note: the table includes only tools that were accessible and registered in five or more studies.

IC Tool	Attitudes	Knowledge	Skills	Internal Outcomes	External Outcomes	Other
Intercultural Sensitivity Scale	17	1	3	0	2	1
Intercultural Development Inventory	-	-	-	-	-	-
Cultural Intellegence Scale/Survey	5	8	0	2	4	0
Cross-cultural Adaptability Inventory	8	2	2	5	0	8
Miville-Guzman Universality Diversity Scale	7	2	0	0	0	6
Generalized Ethnocentrism Scale	22	0	0	0	0	0
Global Perspectives Inventory	16	9	3	1	1	18
Inventory of Cross-cultural Sensitivity	17	6	1	0	0	8
Generalized Ethnocentrism Scale	22	0	0	0	0	0
Multicultural Personality Questionnaire	-	-	-	-	-	-
Total	114	28	9	8	7	41

Discussion

By systematically searching 6 major databases and over 15,000 articles, this study presents one of the largest reviews ever conducted on IC research in higher education. Results highlight new trends in IC research, such as a shift towards interdisciplinary research, as well as gaps in the current literature, which include a paucity of research conducted in South Asia and sub-Saharan Africa and a need for consensus regarding instruments that can be used to evaluate IC.

Results indicate a growing interest in IC research over the past 15 years. A finding that is consistent with previous work mapping publications between 2000 and 2018 (Peng, Zhu, & Wu, 2020). It is likely that this exponential rise in publications is a response to pressure applied on HEIs from external bodies calling for intercultural elements to be included in academic curriculum (Council of the European Union, 2006; European Commission, 2008; UNESCO, 2018). However it is also possible that the rise in publications has resulted from an increasing demand from students who are expecting an international perspective to form part of their studies (Cheng, Adekola, Shah, & Valyrakis, 2016). Unsurprisingly our data, which maps trends up until 2023, shows a major increase in online IC activities since 2020 and the onset of the COVID-19 pandemic although this does not seem to be associated with a major decrease in publications addressing study abroad activities, which one would have expected because of the collapse in international travel. A major point of discussion in higher education post COVID-19 has been on how much of the virtual elements of education introduced during the pandemic will remain permanent (Ludwig, 2022). After an initial increase in 2019/2020, our results show a marked decrease in both online IC activities and in curriculum-based interventions between 2021 and 2022. This will need to be mapped over the coming years to determine if this is a temporary dip or a return to pre-COVID-19 practices.

Students from the discipline of education were the most studied discipline included in IC research. This finding supports results from a systematic review of the Scopus and Web of Science databases and covering the period 2016–2021 (Guillén-Yparrea & Ramírez-Montoya, 2023). The reason why students enrolled in the discipline of education are most highly represented is not necessarily historical as cultural competence appears to have its roots in the health sector (Frawley, Russell, & Sherwood, 2020). There is a tendency, however, for experts in IC development to be more often associated with faculties of education and communication and it can be assumed that they are more likely to recruit students from their own faculty to participate in IC research. While this seems quite natural, the need for developing IC is not less for students enrolled within other disciplines and strategic steps need to be taken to ensure that students outside of the disciplines of education and communication do not fall behind. The field of engineering was poorly represented in IC research included in this review, and we suggest that this requires specific attention. An encouraging new trend arising from the current review is the increase in research combining students from a range of different disciplines. This finding aligns with a global trend towards interdisciplinary research and education which is considered paramount for addressing challenges facing society and the natural environment (Fuchs, 2018).

Previous work has identified North America as the region producing most studies related to IC (Peng, Zhu, & Wu, 2020) and this was very much the case in the present study where publications originating from North America were represented twice as often in research projects as compared to the next most studied group (Europe and Central Asia). Of concern is the gap that can be observed related to the number of studies originating from the Global South. If HEIs are to prepare students for an increasingly global world and promote understanding of the world from different perspectives this issue must be addressed (Yershova, DeJaeghere, & Mestenhauser, 2000). We propose that a failure to include specific regions of the world in the body of IC research is limiting the contextual relevance of research findings as well as the extent to which representatives from the Global South can influence academic discourse related to the topic. The unequal representation of research from the Global South is likely due to heavily skewed funding towards the Global North (Petersen, 2021) and a scarcity of scholars who can conduct or participate in IC research (World Economic Forum, 2024). This inequality has been noted in many areas for example, the World Economic Forum highlights that the number of health researchers in Africa is only 8% of those found in Europe (World Economic Forum, 2024). UNESCO reports that low-income countries spend only 0.23% of their gross domestic product (GDP) on research and development while high-income countries spend 2.74%. To address the gap in the present review lessons should be learned from the many good examples available from other areas of study. This could include development of programmes to support PhD students in the Global South with supervision teams comprising local and international universities (Ayola Akim et al., 2021). Mentorship programs for early career researchers within the area of IC, with careful attention paid to ensuring shared values and power (Candice et al., 2024) and development of open science platforms that convene and co-ordinate regional interest, ideas and resources that are needed to advance IC research in the Global South.

Interventions aimed at fostering the IC of students most frequently focused on pedagogical approaches that were applied within the students’ home institution, although study abroad initiatives were also common. Studies investigating the effects of internationalisation of curriculum reported a higher percentage of positive results than study abroad initiatives. Numerous authors have questioned the impact of study abroad initiatives on IC, suggesting that mobility alone does not enhance students’ competence (Sierra-Huedo & Foucart, 2022). Our review identified studies reporting positive effects of study abroad initiative on IC outcomes (Chang Alexander et al., 2021; Li & Middlemiss, 2022) but also identified studies where no improvement was recorded (Lokkesmoe, Kuchinke, & Ardichvili, 2016; Medina & Kiefel, 2021). Readers are reminded to interpret the summary of positive and negative results in this review cautiously as statistical hypothesis testing was not performed (Lockwood, dos Santos, & Pap, 2019). Results do, however, suggest that a future systematic review and meta-analysis would be warranted to investigate the effect size of specific interventions on IC. Such a study would need to include specific IC evaluation tools as sub-analyses to ensure that the type on instrument being used to evaluate IC does not affect the results.

The most frequently applied method of evaluating IC was to use self-report questionnaires. The wide range of different evaluation tools identified in this study is consistent with previous work (Fantini, 2009; Griffith, et al. 2016; Müller et al., 2020). This finding highlights a lack of consensus regarding the elements of intercultural competence that can, or should, be monitored. A surprising result of the current review was that 74 new IC evaluation instruments were identified as being specifically for individual studies. This may suggest a level of dissatisfaction with current instruments, a question that should be investigated in future research. The finding has important implications for higher educational institutions who are under increasing pressure to record and report the outcomes of IC initiatives. Choosing among the plethora of instruments identified in this review will certainly pose a challenge. Some experts suggest that more than one methodology (e.g. both qualitative and quantitative methods) should be applied when evaluating IC (Deardorff, 2006; Fantini, 2009) and while we agree that this is appropriate within the context of a research study, it does not offer a practical solution for assessing students within the day-to-day educational context. We recognise that there is currently no optimal solution to this problem, and, at present, can only recommend that educators wishing to evaluate the IC of their students familiarise themselves with available tools, identify those which align with the objectives of the assessment and ensure that they understand and acknowledge the selected instruments’ limitations.

Fantini (2009) suggests that a useful approach to selecting an evaluation tool is to first determine which domains of intercultural competence are addressed in the instructional objectives and then determine which tool is best suited to evaluate the domain/s of interest. By mapping items included in the most frequently used IC instruments in this review against components within the Process Model of Intercultural Competence (Deardorff, 2006), our results suggest that there is an overwhelming emphasis on evaluating student attitudes (i.e. respect, openness, curiosity and discovery) in IC research and that skills, internal outcome and external outcomes are rarely investigated. This finding is likely indicative of the complexities of measuring different elements of IC, an issue eloquently addressed by Borghetti (2017). We suggest that that process of mapping items in IC evaluation tools against domains of IC is a particularly useful activity, facilitating understanding of concepts which are being assessed and, importantly, those that are not. We subsequently recommend that mapping exercises be included as an important part of the process when selecting an IC evaluation tool to use in the educational or research context.

Limitations

Throughout this review the authors carefully followed the methods described in the PRISMA-SCR guidelines for scoping reviews. In doing so we applied a systematic approach to mapping the state of evidence related to IC research in higher education. As per recommendations, multiple reviewers were used to identify relevant information sources and to extract data from relevant articles. This process is intended to reduce bias in screening and limit the chance that errors are made. Nevertheless, the authors acknowledge that, with over 15,000 articles screened in this review, there is a likelihood that key articles may have been overlooked during the screening of abstracts and that errors in the data extraction process may have occurred.

To practically manage the vast number of studies available we chose to limit to peer reviewed publications only. Peer review was used as inclusion criterion to try and ensure that only articles of high quality were included. In doing so it must be acknowledged that potentially relevant studies from other sources (e.g. grey literature, books, and theses) may have been overlooked. Similarly, language restrictions imposed in this scoping review may have meant that relevant research that was published in a language other than English, Swedish, Danish or Norwegian was overlooked. It is also possible that the lack of research identified from the Global South may be a result of a language limitations applied when assessing potential articles to include. This issue could be resolved in future studies by partnering with researchers from the Global South, a recommendation strongly supported by the authors.

We chose to classify items included in IC evaluation instruments according to Deardorff’s process model of IC (Deardorff, 2006). To maximise reliability and validity, two authors independently classified all items before comparing results and discussing any conflicting categorisations. The process model of IC is well acknowledged as one of the most recognised models for IC (Arasaratnam-Smith, 2017) although it should be acknowledged that other models describe the components of intercultural competence in different ways and this may lead to different inferences being made about the elements of IC covered in specific evaluation instruments. It is subsequently recommended that future research be conducted to map items in IC evaluation tools against different models of IC. This would provide a deeper insight into the specific IC components that are being measured by different instruments and would also provide a useful resource for educators and higher education institutions who are in the process of selecting relevant tools to use.

Recommendations

Based upon the results of this scoping review the authors suggest that the following actions be undertaken to address gaps that exist in the IC literature.

• Development of clear strategies to support and include scholars from the Global South in IC research initiatives.

• Continued mapping of IC interventions over time. Of particular interest will be to determine if research involving web-based intercultural interactions rises or falls in the post-COVID-19 period.

• Development of initiatives and further research to encourage IC development and research in fields beyond education and communication. While all fields of study are considered important, the field of engineering is poorly represented and should be prioritised.

• Meta analyses to investigate the effect size of specific interventions on IC. This should include sub-analyses performed for specific evaluation tools.

• Mapping of IC evaluation tools against different IC models to better understand the specific concepts of IC they are evaluating.

• Development of support systems and information platforms for educators who need to select interventions and evaluation tools that align with the purpose of a specific IC assessment.

Conclusion

This scoping review confirms that research into IC is receiving mounting attention in higher education research. There is a trend towards studying IC in students representing a range of different disciplines rather than students representing a single discipline. When considering single discipline research, most research investigates students studying within the field of education. The most common types of interventions aimed at fostering students’ IC are pedagogical interventions which are applied at students’ home institutions. During the COVID-19 pandemic there was a trend towards an increasing number of online interventions. Within the studies included in this review there appears to be a gap in research publications focusing on students from South Asia and Africa and an apparent lack of consensus regarding appropriate tools to evaluate IC in student populations. Continued research is required to determine the effects of different interventions on IC outcomes and to understand why so many evaluation tools are being developed and consensus on a select number of tools cannot be reached.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/20004508.2024.2344871

Additional information

Funding

This work was supported by The Swedish Foundation for International Cooperation in Research and Higher Education under Grant [SG20208596].

Notes on contributors

Nerrolyn Ramstrand

Nerrolyn Ramstrand is the Senior Advisor for Internationalisation and a Professor in Prosthetics and Orthotics at Jönköping University. She has a research interest in teaching and learning in international contexts.

Lucie Weisova

Lucie Weissova is an internationalisation at home coordinator at Halmstad University, internationalisation consultant and a doctoral student at Universita Cattolica del Sacro Cuore. Her main research interest is in continuing professional development for internationalisation of the curriculum.

Elisabeth Nylander

Elisabeth Nylander is a research liaison librarian who specialises in supporting evidence synthesis. Her current research interests include new literacies within academic contexts and the development of lifelong learning.

Ann Johansson

Ann Johansson is a lecturer is occupational therapy and the former chair of the International Council at the School of Health and Welfare, Jönköping University. Her main research area is in occupational adaption.