Mitigating the Impact of ChatGPT: Cultivating Professional Identity Through Interactive Oral Authentic Assessments

ABSTRACT

Interactive orals are emerging in prominence as an authentic assessment method, accelerated by an increase in online learning from the COVID-19 pandemic. Limited studies have considered the potential of interactive oral assessments to cultivate professional identity through reflective processes. This paper explores how interactive orals stimulate students to engage in reflective practices and in turn, how critical reflective practices influence the formation of professional identity. Findings revealed interactive oral assessments stimulate high levels of engagement in reflective practices. Goal relevance is critical for encouraging engagement in reflective processes through interactive orals. Reflective processes were stimulated through a double loop learning approach, whereby encouraging students to critically identify their desired identity as a professional in the tourism and hospitality sector, encouraged deeper reflective processes. A conceptual model is proposed along with pedagogical suggestions for further assessment of the efficacy of interactive orals for enhancing graduate outcomes in tourism and hospitality education.

KEYWORDS:

• Professional identity
• interactive oral
• authentic assessment
• reflexivity
• tourism education

Introduction

Interactive oral assessments are emerging in prominence at undergraduate level, with growth accelerated by the COVID-19 pandemic (Arico, 2021). As the higher education sector contended with rapidly scaling up digital take-home assessments, oral examinations saw a resurgence as an authentic and flexible alternative to remote digitally proctored exams (Pradhan & Kreglicki, 2021). However, research on interactive oral assessments at the undergraduate level remains scarce; with isolated examples emerging across the disciplines of business studies, dentistry, education, mathematics, and nursing (Arico, 2021; Sotiriadou et al., 2020). Accordingly, whilst researchers have raised concerns for the fairness, reliability, and validity (Akimov & Malin, 2020), contemporary studies contend that interactive orals’ are authentic, scalable, relevant to employability, compatible with online learning, and reduces the risk of academic misconduct (Akimov & Malin, 2020; Shaeri et al., 2021; Tan et al., 2021; White, 2021). Critically, interactive oral assessments can be completed in-person or online, enabled by synchronous video conferencing applications such as Zoom and Microsoft Teams.

Interactive Orals and Reflection

Interactive oral is an oral examination method in which students’ respond to an assessment task verbally (Joughin, 1998). Interactive oral assessment differs from traditional oral examinations (i.e., presentations and interviews) in its emphasis on the unscripted, synchronous conversation between students and the examiner (Sotiriadou et al., 2020). The interactive component enables the examiner to steer the conversation based on the students’ responses while ensuring the assessment requirements are satisfied, thereby evaluating learning outcomes that are not easily assessed by other methods (Shaeri et al., 2021). This individualized approach to assessment aligns with the “assessment for learning” paradigm that puts learners’ needs at the forefront and emphasizes knowledge application over decontextualized knowledge retention (Black & Wiliam, 2005). The interactive oral exchange is also instrumental in demonstrating a student’s communication, interpersonal, and problem-solving skills (Sotiriadou et al., 2020), which prior research has emphasized are exceptionally important graduate attributes for students entering the tourism and hospitality workforce (Alexakis & Jiang, 2019; Millar et al., 2010).

Depending on the intended learning outcomes and design of the assessment task, interactive orals could promote students’ reflective and critical thinking (Tan et al., 2021). Proponents of interactive orals believe that the effectiveness of interactive orals is best achieved when applied together with authentic assessment task with real-world relevance (Shaeri et al., 2021; Tan et al., 2021). For instance, Sotiriadou et al. (2020) employed interactive orals along with a scenario-based task that resembled an industry practice in a management course. Students were required to reflect and apply the theoretical knowledge learned in the course to solve the scenario as well as effectively articulate the application of knowledge. Tan et al. (2021) investigated the effectiveness of interactive oral assessment across different disciplines and substantiated the professional relevance of the assessment method. When students perceive assessments as more authentic (putting knowledge into practice and being reflective of the process), they are more likely to engage in deep learning as opposed to surface learning, and develop more professional skills (Eddy & Lawrence, 2012; Gulikers et al., 2008; Villarroel et al., 2018). While interactive orals have been applied in business and management subjects (Akimov & Malin, 2020; Sotiriadou et al., 2020), there has been little research on the application of this assessment method in tourism and hospitality education, despite the sector’s strong focus on professional relevance, with limited studies exploring efficacy of interactive orals for enhancing graduate outcomes, such as professional identity.

Professional Identity Development in Higher Education

Previous studies on interactive oral assessments have focused on relevance to employability (Desilets et al., 2021; Gilardi & Lozza, 2009; Mylrea et al., 2017). However, there is growing acknowledgment (particularly in the medical fields) that becoming a professional requires the formation of a professional identity, which is a typically under-researched graduate outcome (Findyartini et al., 2020; Mao et al., 2021; Mylrea et al., 2017). Professional identity is defined as “the self that has been developed with the commitment to perform competently and legitimately in the context of the profession, and its development can continue over the course of the individuals” careers’ (Tan et al., 2017, p. 1505). Professional identity is the complex structure of attributes, beliefs, values, motives, and experiences which infuse to develop a sense of identity in one’s selected profession. The development of professional identity is closely related to values, reasoning ability, clear understanding of responsibilities involved, technical skills, judgment, professional knowledge, and expertise, self-directed learning, critical self-evaluation, self-esteem, job motivation, future perspectives, and reflective practice (Cañabate et al., 2019; Trede et al., 2012).

In today’s complex and dynamic labor market, a notable feature of recent research on employability is the gradual reframing of solving problems with the acquisition of employability skills, to instead more as a relational and socially constructed process that entails graduates to reinforce the relation and commitment to a profession (Engelbertink et al., 2021; Tomlinson & Jackson, 2021; Trede et al., 2012). Specifically in a higher education context, where shaping pre-professional identity has been shown to bridge the relationship between higher education and future employment, facets such as familiarity with working context ideology, proximity in terms of socio-cultural practices to targeted workplaces, networks, and building of social capital, as well as confidence and grounded sense-of-self within the work setting have been found to be key influences on emerging professional identities (Tomlinson & Jackson, 2021). In nursing, higher levels of professional identity have been correlated to higher commitment to a profession and in turn, lower turnover of the workforce (Mao et al., 2021). Critically, professional identity formation among students enrolled in courses that lack compulsory embedded practicums (i.e., tourism and hospitality) have been shown to be relatively weak compared to occupation-specific courses where students often have a clear idealized conception of the profession (i.e., STEM or education) (Tomlinson & Jackson, 2021). Perhaps unsurprisingly, industries like hospitality are uniquely synonymous with high employee turnover (Alexakis & Jiang, 2019). Additionally, a decade since Trede et al. (2012) called for further research on professional identity development in higher education, much remains unclear as to how the development of strong student professional identity can be enhanced or the role of innovative assessment which inspires reflection on facilitating this process (Engelbertink et al., 2021). Indeed, researchers suggest that key parts that are essential for the development of professional identity should include critical reflection, agency, and self-authorship (Engelbertink et al., 2021; Peel, 2005; Trede et al., 2012).

Professional Identity Development Through Reflective Processes

Studies have shown that the optimal conditions for students’ professional identity development should feature curriculum that includes critical reflective practices (Desilets et al., 2021; Gilardi & Lozza, 2009; Mylrea et al., 2017). Accordingly, an increasing number of higher education programs are introducing reflective practices and activities into their curricula – particularly courses or modules with aims of professional identity development (Desilets et al., 2021). Researchers have suggested interactive oral assessments encourage student’s ability for reflective and critical thinking (Tan et al., 2021). Reflection calls for critical self-evaluation, encompassing preexisting beliefs and practices for further action and improvement (Cañabate et al., 2019). Studies have suggested that reflections encourage students to think consciously about their professional identity development – important for analyzing their current identity (i.e., student), their future identity (i.e., what type destination manager they aspire to be), and any experiences of identity dissonance (Findyartini et al., 2020). This process of reconciling different facets of one’s identity is crucial for identity formation and transformational learning, leading to enhanced confidence and reducing likelihood of dissonance between students’ different identity components (i.e., who they are at work, compared to other facets of their life) (Inceoglu et al. 2019; Tomlinson & Jackson, 2021).

Thus, it is clear that prior research points to the importance and potential of interactive orals for exploring the intersection of professional identity and reflective processes. Yet, empirical work combining interactive oral assessment and professional identity is still emerging. Subsequently, the objective of this study is to explore the role and efficacy of interactive oral assessments as a form of assessment for enabling student reflexivity toward empowering formation of their professional identities. The above are operationalized through the following research questions:

RQ1.

How do interactive orals encourage students to engage in reflective processes?

RQ2.

How critical are reflective processes (from interactive orals) in the formation of professional identity?

Method

The interactive oral assessment was introduced in Semester Two of 2021 to a final-year capstone course of a tourism degree in a large Australian university. The aim of the 12-week course was for students to consolidate knowledge and skills acquired across their degree program to address the future challenges and opportunities of the tourism industry. Double-loop learning was embedded in the course curriculum and formed a key component of the reflective process (Cartwright, 2002). Where single-loop learning identifies and rectifies an existing error, double loop learning offers problem analysis and rectification of errors associated with a change of rudimentary beliefs, boundaries, time horizon, goals, and values (Sterman, 1994). Students were encouraged to critique and challenge existing assumptions within the tourism system as well as their own existing assumptions, beliefs, and values as future stakeholders within the tourism, hospitality, and events industry. Correspondingly, students kept a weekly reflective journal based on prompts and reflective questions that formed part of the weekly workshop activities. The reflective journal was designed to empower students to reflect on their perceptions of learning outcomes and apply macro concepts and techniques to their individual contexts and futures within the post-pandemic tourism system. For instance, in the week focusing on increasing automation of the hospitality and tourism workforce, students were prompted to reflect on the impact of automation on their personal futures in the industry. On the week focusing on organizational resilience, students were prompted to reflect on resilience and adaptability of their own career plans. On an operational level, the reflective journals were also emphasized as a key tool for students to refer to at the end of the module in preparation their interactive oral. The reflective journals were submitted as part of their final assessment but not marked. Students were informed that their written reflections did not have consequences on their grades. As this was a capstone module, the interactive oral assessment was carried out at the end of the term to assess students’ learning outcomes for the course and tourism program as well as to facilitate students’ transition from the university into workplaces.

The interactive oral assessment is one of the three assessment items in this course and is weighted 30%. Criteria for the interactive oral assessment was designed in alignment with the program learning outcome (PLO) of establishing students’ professional identity. Marking criteria and assessment guides for the interactive oral included focused on:

1. Understanding of the global tourism sector, with an emphasis on current and future opportunities and threats.

2. Ability to reflect on and analyze your career goals as a future tourism professional, aligning with personal attributes, strengths, weaknesses, beliefs, and values.

3. Ability to research, identify, and develop action plans and strategies for achieving desired career goals.

4. Identification of ethical professional responsibilities.

5. Ability to reflect on aspects of the course and broader undergraduate program.

As reflected in the final point above, students were also encouraged to reflect beyond the module and about their broader university experience. A final non-assessed question probed students to reflect on how and if the interactive oral process enabled them to explore their professional identities. Due to COVID-19 restrictions, all interactive orals were conducted one-to-one online via Blackboard Collaborate. Each session consisted of a 5-minute student presentation followed by 5 minutes of two-way conversation. All interactive orals were recorded.

Analysis

Data included in the analysis were a combination of 90 transcriptions from video recordings of the interactive orals, 90 reflective journals, and reflections of three assessors and authors. Researchers performed open coding to unpack and assign codes to the qualitative data (Cascio et al., 2019). Codes with similar characteristics were clustered into categories. Thematic analysis was then conducted to extract recurrent categories into several themes. Pseudonyms will be utilized to preserve the anonymity of respondents.

Results

Engagement with Reflective Practices

Analysis revealed that interactive orals stimulate high levels of student engagement in reflective processes with the course leaving a “deep impression” [Bindi] on students. Deep engagement in reflective processes, through the weekly journal and workshop activities, allowed students to consider “how to apply the course material to my future” [Cho] with the process of career planning and eliciting professional identity leading to questions surrounding “am I doing the right thing here? It really helped me make many possible scenarios for my future” [Ekval] and realizations like “deeply reflecting on my career development and believe that only relying on tourism skills is not enough, and I need to learn more” [Pepe].

Goal Relevance

Students were particularly enamored with the goal relevance of the reflective process in the interactive oral assessments, noting “Most courses don’t ask you how does learning impact you. This interactive oral actually made me think what did I actually learn … . how does it affect me in my career, what does it do for me? I really enjoyed the reflection.” [Yara]. [Mirri], reflecting on adaptability and resilience in their career, “I haven’t considered it before doing this course. Now I am thinking about it. Covid pandemic make me realise that I was super non resilient in my career plan.”

Cultivation of Professional Identity

Results support that reflective processes contribute to the cultivation of professional identity allowing students to connect areas of the program to desired professional identity, with powerful emotive statements such as the “most valuable lesson I learnt from this course is adaptability … . no matter what I do, I make ethical choices along the way … . I have a responsibility to ensure that the decisions I make promote fair and equal conditions for everyone and everything. I trust that I will never compromise on my values for career success” [Kimana] and “Inequality has always been a pressing issue for me and only to see it expand in recent years has been heartbreaking. As for career goals, I am not sure where I want to take my skills and knowledge in this new normal, but I know that I will be fighting inequality by building community in whatever way I can” [Kaiya]. Reflective processes prompted students to consider how global issues make students feel.

On double-loop learning, [Mirri], who in week 3 of the course noted that they wanted to become a CEO of tour operators in the future, later reflected that “we are stuck in the rat race … system(s) thinking helps to see all of these and proactively adapt to change … double loop learning, be critical, question my assumption to make sure I don’t get stuck in the rat race.” Students were able to connect emergent concepts from the course, such as sustainability with their future career plans and aspirations as a professional, questioning “is what we believe as sustainable really sustainable? Why are we always aiming to tourism growth, increase tourist arrivals? What is the underlying reason for this?” [Mirri]. Students also reflected this in topical issues such as the pandemic, with [Waru], who was also aiming to become a tourism business owner considering “COVID, a big threat to my goal. Currently I see the tourism industry as unstable. However, it may also an opportunity to re-think about how I want to build my tourism business, focus on experience and sustainability.”

Discussion

Argyris and Schon (1996) proposed the theory of double-loop learning as a course of action to deepen learning through reflective and experiential learning in an organizational context. Double-loop learning is actioned through reflection and dialog, integral to building cultures of adaptability and resilience within individual members of organizations (Jaaron et al., 2021; Ritchie & Jiang, 2019). Critically, double-loop learning is designed to encourage critical reflection on beliefs, which underpin the concept of professional identify. Despite this, studies are yet to investigate the potential of double loop learning for stimulating critical reflection of professional identify and the assessment tool to achieve this potential. Aligned with previous research (Desilets et al., 2021), reflective processes have been identified as critical in the cultivation of professional identity. This study advances existing knowledge by demonstrating the efficacy of interactive oral assessments in activating double-loop learning in students’ professional identity formation, by allowing for additional depth to reflective processes. Guided reflection and reflexive processes, especially during a period of major disruption (i.e., COVID-19 providing the opportunity to reshape the future of the tourism industry) can reveal the development of professional identity at a subconscious level, which in turn helps students reshape and negotiate their professional identity formation further (Findyartini et al., 2020; Joseph et al., 2017).

Figure 1 presents a conceptual model which integrates double loop learning into the design of interactive oral assessments, building specific assumptions surrounding beliefs, values and motives, encouraging reflection on goals and the application of systems thinking to encourage students to apply concepts embedded in the course to the concept of professional identity.

Figure 1. Integration of double loop learning process into assessment design.

As exemplified in Figure 1, this research identified that in line with assessment for learning principle (Black & Wiliam, 2005), the design of the assessment task is critical for achieving maximum effectiveness through an interactive oral assessment. In this instance, the application of a real-world scenario or task that is meaningful to the students (e.g., reflecting on their own professional identity) rather than hypothetical scenarios provided for a deeper, more authentic level of engagement among the cohort (Sotiriadou et al., 2020). This research found the interactive oral assessment required students to actively reflect on their learning and link to the task or question. This generated a process of reflection, encouraged at the beginning of the semester through the integration of a reflective journal. This technique was supplemented by a meaningful conversation during the interactive oral session, with post-session reflection and feedback designed to bring reflexivity to the surface, thereby promoting deep learning (Tan et al., 2021) and a long-lasting impression. In light of advances in technology like artificial intelligence capable of passing university-level essay assignments (Purtill, 2023), the importance of embedding these types of authentic assessments that foster critical reflection becomes clear; especially if higher education as a whole wants to ensure the assessment for learning approach is maintained.

Conclusion

As demonstrated in this study, interactive oral assessments provide an alternative and authentic way to measure students’ performance in a course and the graduate attribute of professional identity. Specifically, this paper explored the efficacy of interactive oral assessments for stimulating reflective processes designed to cultivate professional identity. Findings revealed high levels of engagement in reflective practices, with goal relevance critical. Further research is required to understand the potential of interactive oral assessments to enhance conceptually related graduate attributes or for other related contexts such as accreditation purposes. While students from non-English-speaking countries may be disadvantaged in terms of their ability to articulate their ideas freely and fully; because interactive orals are unscripted, interactive orals allow the examiner to steer the conversation to effectively assess students’ learning rather than using a one-size-fits-all approach like written exams or structured interviews. Concomitantly, the flexible, reflective nature of interactive orals could improve accessibility for students such as those who are sight impaired or with hearing loss. In addition, students critically reflect on graduate attributes such as professional identity, this also addresses the fairness issues, as the process is individualized and learner focused. Future studies should compare face to face and virtual interactive orals.

Disclosure Statement

No potential conflict of interest was reported by the author(s).