The effect of the emergency shift to virtual instruction on student team dynamics

ABSTRACT

In spite of the sudden onset of the COVID-19 pandemic, many instructors who used team-based pedagogies shifted them online rather than suspending them entirely, but with limited time and resources. To examine the difference in team dynamics and outcomes for courses in Spring 2019 and Spring 2020 of over 1500 first-year engineering students per semester, Wilcoxon signed-rank tests and random forests method were used. Results show that students reported less improvement in team-member effectiveness, lower psychological safety, and less satisfaction in the semester with the emergency transition. However, students also reported lower conflict. The most important factor predicting project grades shifted from ‘Interacting with teammates’ to ‘Having relevant knowledge, skills, and abilities’ amid the emergency shift, accompanied by a reduction in team interdependence. In spite of the collection of data during an emergency transition, the foundation of face-to-face interaction before moving to virtual cooperation represents a useful contribution to research that has focused exclusively on virtual learning circumstances.

KEYWORDS:

• Teamwork
• team dynamics
• team-member effectiveness
• virtual learning
• CATME
• COVID-19

1. Introduction

The ability of engineering students to function effectively in teams has been emphasised as an essential learning outcome in education (ABET 2019) and a critical competency in the job market (Loughry, Ohland, and Woehr 2014). Moreover, providing team-based learning experiences in engineering education benefits students, as they will be able to develop collaboration, communication, and conflict management skills necessary for their success as future engineers. Most research investigating teamwork and team-based learning has focused on face-to-face teams (Gelbard and Carmeli 2009; Humphrey and Aime 2014; Paul, Drake, and Liang 2016), or virtual teams of various forms (Huber 1990; Jarvenpaa, Knoll, and Leidner 1997; Kramer, Shuffler, and Feitosa 2017; Schaubroeck and Yu 2017; Schmidtke and Cummings 2017). Fewer studies have interrogated blended-mode team collaborations that include both physical and virtual interactions, or even HyFlex (a hybrid course permitting flexible learner attendance) approaches in higher education (Bosman, Wollega, and Naeem 2022; Magana et al. 2022).

The emergency shift to virtual instruction due to COVID-19 unanticipatedly formed a stress test for everyone, which left students and instructors with little opportunity to prepare for the transition. The second half of the Spring 2020 semester witnessed a scenario off guard in which educational institutions, instructors, and students were almost all involved in exclusively virtual instruction and learning. The sudden shift in instructional modality impeded the implementation of best practices of teamwork training based on traditional in-person classroom interactions (Krishnakumar et al. 2022; London, Douglas, and Loui 2022; Thomas, Patel, and Magana 2021). The transfer to virtual modality posed significant challenges for instructors (Shekh-Abed and Barakat 2022). Instructors were not prepared for virtual instruction, making it challenging for them to manage virtual teams in addition to the other effects of the transition. For example, many instructors had to adapt courses to be held virtually within a short period, revising syllabi and examinations and other assessments accordingly, selecting and learning to use tools to support virtual instructions, etc. (Bosman, Wollega, and Naeem 2022; London, Douglas, and Loui 2022). The challenges were amplified in courses using team-based pedagogies, where instructors lost the opportunity to observe student team dynamics in person (Krishnakumar et al. 2022; Magana et al. 2022).

The transition to virtual instruction was challenging for students as well. The disruptive impact of COVID-19 was pervasive in the student experience – not just in this first-year class and not just in their team experiences, as students were under pressure to adjust their learning modes, including the effects on mental health based on all kinds of accidental events related to the health state of themselves and their family, source of income, etc. (Cao et al. 2020; Oliva-Cordova et al. 2022; Thomas, Patel, and Magana 2021). A study exploring students’ perception of online learning during the stay-in-place orders showed that students preferred face-to-face instruction due to the unpleasant online learning experiences and explained that online learning was not only difficult but also lacked various supporting resources (Patricia 2020). In addition, Patricia (2020) also discovered that cognitive engagement, including knowledge, concentration, engagement, attendance, and interest among students, decreased after stay-in-place orders. Panchal et al. (2020) reported that the COVID-19 pandemic has negatively impacted a great number of people’s mental health and may significantly influence people’s psychological attributes in normal work. All those work-life stressors complicated and interacted with individual-level stressors, such as concerns about own health, and overwork and fatigue (Tannenbaum et al. 2021). As crisis-induced difficulties functioned on each individual, but it was the team which must collectively manage all of these struggles at once and change the team dynamics (Wildman et al. 2021).

Studies have provided early evidence of the potential impact of COVID-19 on collaborative learning and teamwork. The crisis-induced switch from face-to-face to virtual teamwork negatively impacted teamwork dynamics (Ruparell 2021; Wildman et al. 2021), teamwork effectiveness (Krishnakumar et al. 2022), team performance (Wildman et al. 2021), and interaction and communication (Gutierrez et al. 2022; Wildman et al. 2021) in general. Krishnakumar et al. (2022) illustrated that a lack of relationships with others and an insufficient mechanism of building and sharing knowledge through interactions hindered student success in suboptimal environments in first-year engineering courses. Many students had to engage in team-based projects regardless of their familiarity and facility with tools to support virtual collaboration and communication even across geographical dispersion, time zone, and other divisions intensified under the COVID-19 breakout semester (Alberto Espinosa et al. 2003; Krishnakumar et al. 2022; Magana et al. 2022). Furthermore, when engaged in team projects, students might feel unsafe producing work due to forced adjustment to the online learning and teaming environment if the instruction was not oriented to produce a welcoming and encouraging space for social presence and collaborative learning. Magana et al. (2022) demonstrated that the use of cooperative learning pedagogy with Hyflex accommodations and conflict training could provide a comparable alternative to residential learning.

Taken together, however, compared to the study on in-person teamwork and well-planned virtual teamwork, relatively little is known about how student teaming experiences, particularly team dynamics, are influenced by the change in learning environment due to the emergency transition to virtual instruction. Thus, the research question of this study is how and the extent to which first-year engineering students have different teaming experiences related to the emergent shift to virtual instruction as measured by student team dynamics, team-based task performance, and team satisfaction, compared to the same course offered a traditionally residential instruction in a previous semester. Specifically, this observational study quantitatively compares student team dynamics and outcomes before and during the COVID-19 emergency transition to virtual teamwork to understand its unexpected and great impact on learners’ work processes and team performances in team-based courses. Although this study was situated in a special context unlikely to happen again, the circumstance provided a unique stress test for students and instructors to engage in team-based learning courses, which could enrich their understanding of team phenomenon and perceptions, especially in the virtual mode. Further, we provide insights and suggestions on how to improve students’ virtual team experiences and their virtual collaborative learning environments for post-pandemic instruction.

2. Literature review

In this section, we provide relevant literature on factors related to virtual teams, team dynamics and team satisfaction. We define team dynamics in this study as the temporal state of psychological perceptions and behavioural skills that team members possess when working within a team towards a common goal. In this study, we operationalise team dynamics as team interdependence, team conflict, psychological safety, and teamwork behavioural skills in terms of the Comprehensive Assessment of Team Member Effectiveness (CATME) behavioural-anchored rating scale.

2.1. Virtual teams

Virtual teams share both similar and different yet more demanding characteristics of face-to-face teams. Many instructors sought to develop virtual-team-based pedagogy to maintain teamwork training for decades. Virtual teams are different from in-person teams in the sense that virtual teams are composed of geographically dispersed people who mainly rely on technology for communication and collaboration to work towards their shared goal across boundaries of distance, time, and other divisions (Alberto Espinosa et al. 2003). On one hand, virtual teams have advantages, such as effectively utilising expertise and resources to diversify information and values and being more creative due to potential demographic diversity (Kiesler and Cummings 2002). On the other hand, virtual teams face typical challenges of communication and collaboration, featured by a lower level of media richness, a lower level of team member engagement, barriers to creating trust and a shared mental model (the team’s cognitive structures of the task, interactions and environment) of task division and responsibility, and a higher level of social distance among members (Dulebohn and Hoch 2017). With a lack of in-person presence, team members are often less aware of the team’s status and progress towards preset goals, and team dynamics may be impaired, which further hinder the growth opportunities for team members to uncover and resolve conflicts virtually, and develop team potency and shared motivation (Dulebohn and Hoch 2017; Zaccaro and Bader 2003; Zigurs 2003). Moreover, research has reached a consensus that virtual teams tend to need more time to make decisions but are more likely to generate higher quality of ideas for solving problems relative to face-to-face teams (Schmidtke and Cummings 2017). However, Krishnakumar et al. (2022) argued that the virtual learning cannot well capture the circumstances of emergency transition to virtual instruction due to different intentions and preparations for the class settings.

Virtual teams have also been noted to rely on computer-mediated communication (CMC) that may lessen the perceived warmth of team members’ communication, setting barriers for team members to develop interpersonal relationships in online teams (Walther 1992). Compared with in-person teams, virtual teams rarely support social cues that form a method of dealing with team functioning, reducing the possibility of synchronous communication among team members due to the unfamiliarity with the type and number of communication channels available (Montoya-Weiss, Massey, and Song 2001; Reisetter and Boris 2004; van Tryon and Bishop 2009; Zielinski 2000). For example, Pazos et al. (2019) found that the use of a virtual collaboration tool suite in engineering education collaborative student teams did not significantly predict team satisfaction. Virtual teams often suffer social connectedness with reduced sensory channels of communication, which increases psychological distance between team members (Jacques et al. 2009). Virtual teams also have difficulty overcoming barriers to achieving team effectiveness, as the higher the degree of virtualness, the greater the complexity that team members need to confront before reaching team effectiveness (Cohen and Gibson 2003; Marlow, Lacerenza, and Salas 2017). Despite the typical challenges in virtual teams, the use of virtual teams continues to grow in various settings (Cordery and Soo 2008), such as global collaboration (Maznevski and Chudoba 2000) and education institutions (Chen et al. 2008). Meanwhile, research also empirically challenged the findings that emerged from experimental studies that virtual teams were inferior to face-to-face teams and argued that familiarity with virtual collaboration and tool usage might mitigate the negative impact of virtual learning (Purvanova 2014). Despite the difference with the intentionally designed virtual team-based learning, this study contributes to the body of literature of virtual team as it is situated in a natural educational context shared some characteristics of virtual collaborative learning. Besides research on the practice of virtual teams, the topic of how team dynamics are interrelated with each other and how team dynamics influence team outcomes are central to the study.

2.2. Team dynamics

Team dynamics are crucial to teamwork and team experience, so as to teams’ success (Delice, Rousseau, and Feitosa 2019). Positive team dynamics stimulate learning and creativity within student teams, where creativity fosters teamwork and improves overall team performance in the engineering field (Gelbard and Carmeli 2009). On the other hand, negative team dynamics hinder important team outcomes, including team creativity (Chang 2011) and effective communication (Henderson 2008). Team dynamics also reflected team awareness, providing team members with information about different aspects of team development that helped teams achieve the best results (Oemig and Gross 2007). With the emergency transition to online learning and teaming, virtual teams modelled a common setting for organisations and institutions (Magana et al. 2022; Patricia 2020). In the following subsections, we further discuss three principal psychological attributes of team dynamics: team interdependence, conflict, and psychological safety, as well as their relationships with virtual team experiences of students.

2.3. Interdependence

Interdependence is a crucial aspect of effective teamwork, and it has been categorised into three types: goal interdependence, task interdependence, and outcome interdependence (Campion, Medsker, and Higgs 1993). Campion, Medsker, and Higgs (1993) showed that three types of interdependence were positively related to team effectiveness and overall performance in traditional in-person teams. Another study on interdependence in in-person teams also showed that accountable interdependence contributes to team members’ positive attitudes toward teamwork (Ruiz Ulloa and Adams 2004). Moreover, the three types of interdependence have also been found to be positively correlated with team effectiveness in virtual teams (Hertel, Konradt, and Orlikowski 2004). Evidence from the literature indicated that interdependence might take different forms in residential and virtual teams (DeSanctis, Staudenmayer, and Wong 1999). Also, students who experienced transition from in-person to virtual teams might have different perceptions of interdependence as their team dynamics might also undergo restructuring. Despite the deficiency of face-to-face interactions that may negatively affect team interdependence in virtual teams, many scholars remained optimistic that virtual teams can still develop sufficient team interdependence if they are able to utilise technology advantageously to facilitate team functioning (Kirkman and Mathieu 2005). Furthermore, virtual teams that could leverage interdependence, and virtual teamwork had a better potential to succeed in the modern environment with teams of complex structures (Maynard et al. 2012). However, if not managed well, online teams would likely suffer from experiencing difficulties in maintaining interdependence (Gibson and Manuel 2003), which would diminish the advantages that interdependence possesses in fostering communication and cooperation among team members in online teams, likely leading to team conflict (Somech, Desivilya, and Lidogoster 2009). Particularly relevant to this work, interdependence has been found to moderate other team dynamics in a study using CATME data (Thomas et al. 2019). Specifically, dyadic viability – whether a student wanted to work with a particular teammate again – was more related to whether students liked their teammate when interdependence was high, and more related to their teammate’s competence when interdependence was low.

2.4. Team conflict

Team conflict, broadly defined as discrepant views among team members (Jehn and Bendersky 2003), can be categorised into relationship, task, and process conflicts (Jehn 1995). With team members working across geographical, time, and space boundaries, virtual teams are more likely to experience team conflict (Kankanhalli, Tan, and Wei 2007). Furthermore, virtual teams often encounter more difficulties in practicing conflict management, thus making it vital for conflict to be effectively controlled to stimulate collaboration and improve team performance (Furst, Blackburn, and Rosen 1999). With the absence of spontaneous communication, a moderator effectively detecting and resolving conflicts through the facilitation of shared identity and context, virtual teams were likely to undergo more hardships in managing team conflict (Hinds and Mortensen 2005). Virtual teams also tended to be more vulnerable in the face of conflict as the technology-mediated communication needed by virtual teams was deficient in social presence and interactivity, impeding the conveyance of multiple cues and lowering the level of conflict management (Zach 1993). Research also pointed out that in-person meetings were necessary for resolving conflicts in teams (Dubé and Robey 2009). Therefore, considering that virtual teams suffered from reduced face-to-face interaction (Hertel, Konradt, and Orlikowski 2004), especially in the case of the forced transition to online learning due to COVID-19, they were more likely to experience conflicts. If managed poorly, conflict could be detrimental to overall team performance and success (Barki and Hartwick 2001). Closely tied with and negatively associated with team conflict, psychological safety is a team dynamic that needs to be assessed when virtual teams are evaluated (Johnson and Avolio 2019).

2.5. Psychological safety

Psychological safety refers to a consensus reached by team members that the team is safe for interpersonal risk-taking (Edmondson 1999). Research showed that psychological safety applied to virtual teams as well (Cordery and Soo 2008). As team psychological safety develops from team members’ shared beliefs, it should converge in a team and facilitate overall learning behaviour. Moreover, research on psychological safety within traditional in-person teams revealed that team psychological safety was a form of team dynamic that significantly minimised team members’ concerns of embarrassment (Edmondson 1999), stimulated team learning behaviour (Edmondson and Lei 2014), and provided team members with more confidence to take risks (Van den Bossche et al. 2006). In virtual teams context, Gibson and Gibbs (2006) discovered that a psychologically safe teamwork environment mitigated the negative effect of working virtually as less efficient and productive. In addition, Ortega et al. (2010) revealed a positive relationship between psychological safety and learning-oriented interactions in virtual teams and found that increased levels of psychological safety facilitated virtual team learning behaviours. However, the virtual team setting often makes the team dynamic of psychological safety less effective and less feasible (Gibson and Manuel 2003). Together with conflict, psychological safety affects team members’ satisfaction with their team (Johnson and Avolio 2019).

2.6. Teamwork behavioural skills

In addition to psychological constructs of team dynamics, including interdependence, team conflict, and psychological safety, team members rely on teamwork behavioural skills to collaborate with each other and influence team dynamics. Both psychological and behavioural aspects of team dynamics have been found to link to team task performance, as well as team satisfaction (Vegt, Emans, and Van de Vuert 2001). Schmidtke and Cummings (2017) proposed that with the increased level of virtualness of team context, a team’s shared mental models became more complex, which prevented the effectiveness of particular teamwork behaviours: mutual performance monitoring, backup behaviour, and adaption based on their literature synthesis. Research also demonstrated that the commitment disparities among teammates interfered with their goal alignment and communication behaviours (Manata et al. 2021). Literature also documented the team member performance issues in emergent transition to virtual team learning, for instance, perceived increased forgetfulness, increased procrastination, exacerbated issues surrounding social loafing, and changes in communication patterns (Wildman et al. 2021). Teamwork behavioural skills occur throughout whole lifetime of team task, interplay with other team dynamics constructs, and regulate team performance (Mcewan et al. 2017). Teamwork behavioural skills are associated with other team dynamics and outcomes. For example, teamwork behavioural skills have been found to be positively correlated with interdependence (Hertel, Konradt, and Orlikowski 2004). A lack of teamwork behavioural skills leads to increased team conflict, less psychological safety, and less interdependence (Beigpourian et al. 2019). Team dynamics, in terms of interdependence, team conflict, psychological safety, and team skills, affected team satisfaction and task performance (Vegt, Emans, and Van de Vuert 2001).

2.7. Team satisfaction

Satisfaction in teams is often categorised into job satisfaction and team satisfaction, where job satisfaction refers to a team member’s overall satisfaction with their own work, and team satisfaction refers to a team member’s perception of working with other team members as a team (Vegt, Emans, and Van de Vuert 2001). In virtual teams, satisfaction has been found to be positively influenced by interdependence and negatively influenced by conflict (Hinds and Weisband 2003), and team members’ satisfaction increases as time passes in online teams (Chidambaram 1996). Satisfaction is crucial to the development of virtual teams, as job satisfaction is a factor that is directly associated with their functioning (Sweeney and Boyle 2005). Furthermore, when team members’ level of satisfaction increases, each team member will be more likely to perform better and want to remain as a member of the team (de la Torre-Ruiz, Ferrón-Vílchez, and Ortiz-de-Mandojana 2014), enhancing the need to facilitate satisfaction in online teams (Robert and You 2018). Despite the importance of satisfaction in virtual teams, research revealed that the virtual teamwork environment could at times be unsatisfying for team members (Ortiz De Guinea, Webster, and Staples 2012), and generally lower the level of satisfaction in team members’ interactions (Warkentin, Sayeed, and Hightower 1997), making it increasingly necessary to monitor student team members’ satisfaction. Therefore, in this work, we use team satisfaction to represent student team members’ affective outcomes.

2.8. Summary

Based on the literature reviewed regarding team dynamics in traditional in-person and virtual teams, and the increasing demand for virtual teams brought by the unexpected COVID-19 pandemic, it is important to study how the virtual or mixed teamwork environment affected team dynamics, satisfaction, and collaborative learning in team-based projects. The result of this study will add to our knowledge of how team dynamics have changed after the transition to virtual teams due to the COVID-19 pandemic and complement early findings related to student team experiences. Moreover, this study will add to the resources and strategies available to instructors and students on how to facilitate and participate in effective collaborative teamwork in a virtual teaming environment or under a stress test.

3. Conceptual framework

Our conceptual model was developed based on the taxonomy of teamwork effectiveness (Wei et al. 2020) and the conceptual framework of team effectiveness proposed by Lurey and Raisinghani (2001). We customised our conceptual model by focusing on the behavioural, affective, process, and task performance in teams given our research design. As illustrated in Figure 1, we highlighted the hypothesised direct effects of the emergency transition to online learning by conducting virtual teamwork on all types of team dynamics and performance. Through the developed conceptual model, we attempt to explore what effects an emergent shift to virtual instruction may have on team dynamics, course performance and team behaviours and satisfaction.

Figure 1. Conceptual framework of team-member effectiveness and team dynamics.

As discussed in the literature review section, behavioural, affective, process, and task performance are interconnected. Teamwork behavioural skills occur the whole lifetime of team task and regulate team performance (Mcewan et al. 2017). The behavioural performance of individual team members was measured via the CATME behaviourally anchored peer evaluation instrument and reflects students’ perceptions of the team contributions (Ohland et al. 2012). We operationalised teamwork behavioural skills as the five-dimension CATME behaviourally anchored rating scales in five dimensions: contributing to the team’s work, interacting with teammates, keeping the team on track, expecting quality, and having relevant knowledge, skills and abilities (Ohland et al. 2012). Team process performance means the interpersonal process of team cooperation and team conflict during team members’ interaction and coordination (Nederveen Pieterse et al. 2013; Rupert et al. 2019; Sakuda 2012). In this study, we operationalised team process performance as team dynamics in terms of team interdependence, conflict, and psychological safety. Affective performance in this study refers to team member’s satisfaction towards the team and perceptions of the team’s future viability (Lewis 2004; Robertson, Gockel, and Brauner 2013). Task performance took many forms in existing studies, which emphasised the quality of decisions, outcomes, or deliverables of given group tasks, such as simulation games’ team scores, sales’ profit, assets’ return, sports scores, internal or external team evaluation or rating and clients’ satisfaction (Bachrach et al. 2019). Individual final grades (as percentages) and team project grades (as percentages) were used to model the task performance of team members in this study.

Lying at the core of the conceptual model shown in Figure 1, are the process performance of three constructs: team interdependence, conflict, and psychological safety, and teamwork behavioural skills measured by CATME’s five dimensions. The task performance, or scores of the students’ team-based projects and individual grades, and the affective performance represented by satisfaction were measured at the end of semesters. The effects on the target variables caused by the shift to online instruction in Spring 2020 semester were measured in comparison with the outcomes of the same variables in the Spring 2019 semester when the instructional environment was not disturbed by any emergent situation.

4. Methods

4.1. Participants, data collection and preparation

Data were collected in a 16-week first-year engineering course at a large Midwestern public research-intensive university in Spring 2019 (a fully residential semester with 1679 observations) and Spring 2020 (the emergency transition semester with 1777 observations). The course was required for first-year engineering students, and they were assigned to teams with a target size of four persons. The collection and use of data were granted by this university’s institutional review board with informed consent waived. The data were collected using the CATME system for all self-reported rating scores and demographics as well as from instructors for all course and team project grades. The self-reported demographics of the two cohorts are summarised in Table A1, including gender, race, first semester GPA, academic level (based on accumulated credit hours), and a binary variable indicating whether the student is international or domestic. The demographic distributions of the two cohorts were similar. Figure 2 visualises the timeline of this course with key events. Teams were assigned in week 2 of the course, and intensive team-based assignments and projects started around week 8. Four rounds of self- and peer evaluations of the CATME five-dimension teamwork behaviour survey, as well as survey questions regarding team dynamics and satisfaction, were administered in weeks 8, 12, 14, and 16, via the CATME web interface (Ohland et al. 2012). Team interdependence was only measured in week 8, while team conflict and psychological safety were measured in weeks 12 and 14. Team satisfaction was surveyed in week 16. Spring break for this institution occurred in week 10 in the Spring 2020 semester, and the emergency transition to virtual instruction was effective starting in week 11. Survey completion comprises a small portion of students’ final grades, and all surveys had a high response rate (over 85%). The final grades of each student and project grades of each team were gathered and reported as percentages.

Figure 2. Course timeline with key events.

The self and peer ratings were collected to motivate students to contribute to their teams, demonstrate teamwork behavioural skill expectations, evaluate contribution, and provide feedback for teamwork improvement (Ohland et al. 2012). By rating themselves, individuals of a team can reflect on their own teamwork behaviours and experiences and learn how to contribute better to the success of their team. Self- and peer ratings are closely and significantly related to task and affective outcomes. Instructors relied on student self- and peer evaluations to determine students’ grades that reflect the degree of their contributions to the team. Self and peer evaluations can be used to discourage social loafing, promote more positive attitudes towards teamwork, and foster greater satisfaction with team members’ contributions (Aggarwal and O’Brien 2008; Chapman and Van Auken 2001; Pfaff and Huddleston 2003). Several studies have found self and peer ratings have good predictive validity for various performance criteria (Conway and Huffcutt 1997; Viswesvaran, Schmidt, and Ones 2005).

In addition to the self and peer evaluations, which measured team-member effectiveness and teamwork behavioural skills (Ohland et al. 2012), a number of published instruments were administered within the CATME system to measure team interdependence (Vegt, Emans, and Van de Vuert 2001, with minor modifications), team conflict (Jehn and Mannix 2001), psychological safety (Edmondson 1999), team satisfaction (Vegt, Emans, and Van de Vuert 2001, with minor modifications). The instruments are shown in Appendix B in detail. For CATME peer-rating scores, ratings from all raters for a ratee were aggregated by averaging ratings from all peers. If fewer than two peer ratings were available at a single time point, the scores were discarded from further analysis. All team dynamics and satisfaction scales were constructed as 5-point Likert scales, except for psychological safety, which has a range of one to seven.

4.2. Empirical approach

We hypothesised that the first-year engineering students had different teaming experiences related to the emergent shift to virtual instruction as measured by student team dynamics, team-based task performance, and team satisfaction, compared to the same course offered a traditionally residential instruction in a previous semester. We conducted a series of Wilcoxon signed-rank tests (Wilcoxon 1945) to compare the longitudinal development of student perceived self- and peer evaluations of behavioural team effectiveness within the transitioned semester, and to contrast scores of team dynamics, task performance, and satisfaction over the two semesters. In addition to the sequence of Wilcoxon tests, we also conducted random forest analysis (Liaw and Wiener 2002; Strobl et al. 2007) to inspect the relative importance of all predictor variables (student perceived teamwork behavioural skills and team dynamics measurement) on outcome variables (task performances and team satisfaction) for both semesters. By doing so, we could compare the most important factors of team dynamics on student team satisfaction and task performance.

Specifically, we used the relative importance measure within the random forest for estimation, where the relative importance of a given explanatory variable is defined as the relative prediction accuracy loss when that variable is excluded from the dataset and random forest model (Hapfelmeier et al. 2014; Strobl et al. 2007, 2008). The advantage of the random forest method is that it reduces the chance of overfitting (Liaw and Wiener 2002). The variable importance measure within the random forest method has several advantages, as shown by previous studies, including that it can be used in datasets with missing values, mixed categorical and continuous variables, and near-collinearity issues (Hapfelmeier et al. 2014; Strobl et al. 2007, 2008). Therefore, the random forest-based variable importance measure has been used and found to be informative in multiple educational research studies (e.g. Beaulac and Rosenthal 2019; Mendez et al. 2008; Tan, Main, and Darolia 2021). For example, Mendez et al. (2008) exploited the random forest method to identify explanatory variables predicting student persistence in science and engineering majors. Following these existing studies in the educational research literature, we also employed a similar methodology in our applications to identify explanatory factors that were most important in predicting student team learning outcomes across two semesters.

5. Findings

5.1. Students reported lower self- and peer ratings of teamwork behavioural skills in the emergency transition semester compared to the residential semester

As shown in Figure 3, for all six subplots, the red line (residential) generally lies above the green line (transition to virtual), which means that across multiple dimensions, students in a fully residential semester (Spring 2019) rate their peers slightly higher than in the emergency transition semester (Spring 2020). We averaged the scores of all five dimensions to study overall team behaviour and used Wilcoxon signed-rank tests to compare the associated same round of survey across the two semesters and to compare the adjacent two rounds of survey within the same semester. Results showed that except for the first round of evaluation, the peer rating results for the rest three surveys of the Spring 2019 semester were significantly higher than those of the Spring 2020 semester. Among the disaggregated dimensions, that pattern was observed for Dim C (Contributing to the team’s work) and Dim I (interacting with teammates). For Dim K (keeping the team on track), only the second and fourth rounds of the evaluation showed significant differences across the two semesters. Dim E (expecting quality) and Dim H (having relevant KSAs) additionally exhibited differences even in the first round of measurement. The descriptive summary of self- and peer-rating scores for both across all dimensions and individual dimensions could be found in Appendix Table A2.

Figure 3. Averaged peer-rating results for each evaluation round in both Spring 2019 and Spring 2020 semesters, aggregated and for each dimension.

Note: The red and green characters and lines represent results of Wilcoxon signed-rank T-test across rounds of evaluation for Spring 2019 semester and Spring 2020 semester respectively. Asterisks or ‘na’ represents significance of the Wilcoxon test where the effect size of Cohen’s d contained within the parenthesis. The blue characters indicate the significance and effect size of Wilcoxon signed-rank test across the two semesters for each round of evaluation. * p < 0.05, two-tailed; ** p < 0.01, two-tailed; *** p < 0.001, two-tailed

Self-rating of teamwork behaviours followed a similar overall trend to the peer rating results discussed above; students in the Spring 2020 semester tended to rate themselves lower than those in the Spring 2019 semester. The patterns across all dimensions in the aggregate and individual dimensions of self-rating results matched with the peer-rating counterparts, and are shown in Figure 4. The average self-rating scores assessed by each survey over the two semesters were also organised in Table A3 in the Appendix.

Figure 4. Averaged self-rating results for each evaluation round in both Spring 19 and Spring 20 semesters.

Note: The red and green characters represent results of Wilcoxon signed-rank test across rounds of evaluation for Spring 2019 semester and Spring 2020 semester respectively. Asterisks or ‘na’ represents significance of the Wilcoxon test where the effect size of Cohen’s d contained within the parenthesis. The blue characters indicate the significance and effect size of Wilcoxon signed-rank test across the two semesters for each round of evaluation. * p < 0.05, two-tailed; ** p < 0.01, two-tailed; *** p < 0.001, two-tailed

It was also noticeable that starting from the second round of evaluation (PE2), students were inclined to provide higher peer- and self- ratings of teamwork behavioural skills compared to the first found (PE1) whereas the difference between the second round to the third round (PE2 vs. PE3) and between the third round and the fourth round (PE3 vs. PE4) was not significant or significant with little effect size. From both Figures 3 and 4, we note that, throughout the course cycle, rating scores in the fourth round of evaluation were significantly higher than those in the first round of evaluation for all cases, which implied that students still perceived improvement in their teamwork behaviours even if the major collaboration pattern changed from offline to online.

5.2. Students reported lower psychological safety and lower team satisfaction, but less team conflict

Wilcoxon tests were conducted to examine if each of the aggregated team performance measurements collected in the Spring 2019 semester were significantly different from those collected in the Spring 2020 semester. The statistics of those measurement results were summarised in Table 1. The average scores of team interdependence, conflict, and satisfaction gathered in the fully residential semester were significantly higher than those collected in the emergency transitioned semester, with effect sizes of Cohen’s d ranging from small (0.14) to medium (0.47) (Sawilowsky 2009). Students in the Spring 2019 semester reported more interdependence and satisfaction, but also reported more conflict than students in Spring 2020. Students in the Spring 2019 semester also reported a much higher degree (about two out of seven points) of psychological safety with large effect sizes, measured in both the second and third evaluations. Nevertheless, it is important to note that despite these difficulties, students on average received slightly higher individual scores and team project grades in the Spring 2020 semester with relatively small effect sizes.

Table 1. Wilcoxon test statistics of cross semester comparison for team dynamics, team satisfaction, and course grades.

Measured Items	Mean in SP19	Mean in SP20	Sig	Cohen’s d
Interdependence (PE1)	3.281	3.117	***	0.27
Team Conflict (PE2)	1.441	1.279	***	0.47
Psychological Safety (PE2)	6.035	4.000	***	3.80
Psychological Safety (PE3)	6.128	3.991	***	3.85
Team Satisfaction (PE4)	4.337	4.229	***	0.14
Individual Course Grade	86.191	88.325	***	−0.26
Team Project Grade	89.426	90.268	***	−0.11

Note: * p < 0.05, two-tailed; ** p < 0.01, two-tailed; *** p < 0.001, two-tailed.

5.3. The most significant factors influencing satisfaction and task performance varied across the two semesters

As introduced earlier, our explanatory variables were the three team dynamics of interdependence (measured in PE1), conflict (measured in PE2), and psychological safety (measured in PE2 and PE3) as well as all five individual CATME teamwork effectiveness dimensions measured in all rounds for both peer- and self- rating results. Our outcome variables included individual course grades, team project grades, and team satisfaction. The relative importance of an explanatory variable was determined as the loss of prediction power when that variable was excluded from the dataset (Strobl et al. 2007). Note that relative importance measures are calculated independently for each variable, and their magnitudes are relative to one another. We scaled these measures so that the sum of the importance values equals 1, allowing for easier comparison and interpretation of the results. We also ran parallel analyses for data collected in Spring 2019 and Spring 2020 semesters.

The results from the relative variable importance analysis via the random forest method were presented in Figure 5. The results for project grades and team satisfaction are presented in Panels A and B, respectively. The results for individual course grades were unremarkable due to having similar important factors and relative important scores and are discussed below but do not merit a figure. Following a previous study (Tan, Main, and Darolia 2021), we arbitrarily kept only the top 10 most important measures, but our qualitative conclusions were not affected by reasonable alternative approaches. In each panel, we focus on the comparison between the Spring 2019 and Spring 2020 results. Note that the error band stands for the 95% confidence interval, generated with a bootstrap procedure with 100 replications.

Figure 5. The most significant factors predicting team outcomes in each semester.

Note: The error bar represents the 95% confidence intervals of the relative importance of a given factor.

The important factors for predicting individual course grades were qualitatively very similar in Spring 2019 and Spring 2020 and focused on peer-ratings of contributing to the team’s work and having relevant KSAs. Since the individual course grades included individual assessments such as exams, it was unsurprising that the top factors were dominated by effort and knowledge, skills, and abilities.

On the other hand, the important factors for predicting team project grades were quite different between Spring 2019 and Spring 2020, as shown in Panel A. Dimension E, Expecting quality, was important for both semesters, but dimension I, Interacting with teammates, no longer topped the list of the most significant factors in predicting project grades in Spring 2020, the emergency transition semester. The analysis results demonstrated that Having the relevant KSAs, Expecting quality, and Keeping the team on track were the top key factors. The change implied that student team members might now adopt a more individualised working mode to complete team projects, which might be dramatically different from their approach when face-to-face collaboration is feasible. This conjecture is supported by the reduction in interdependence observed in Spring 2020, noting that competence is more important in teams with less interdependence (Thomas et al. 2019).

From Panel B, the key factors predicting team satisfaction were also different between the two semesters. In Spring 2020, the emergency transition semester, team conflict in PE2 and self-rating of Dim E in PE2 and PE4 were among the top three significant factors predicting team satisfaction. However, in Spring 2019, the fully residential semester, psychological safety measured in PE3 and PE2, along with team conflict in PE2 mostly contributed to the prediction of team satisfaction for the emergent transition semester. The results might suggest that although the conflict was lower on average in Spring 2020, that teams that experienced conflict over the quality of submitted work were particularly at a disadvantage during virtual instruction.

6. Discussion

The sudden onset of the COVID-19 pandemic generally caught instructors and students off guard, forcing them to switch from the traditional face-to-face mode to one of virtual instruction (Bosman, Wollega, and Naeem 2022; London, Douglas, and Loui 2022). This transition posed great challenges for instructors and learners. This empirical comparative research investigated how first-year engineer students’ teamwork experiences were affected by the emergency transition to virtual learning caused by the COVID-19 pandemic. Our study yielded three primary findings. First, students reported less improvement in teamwork behavioural skills from both themselves and their peers. Second, students reported less interdependence, conflict, psychological safety, and satisfaction during the semester transitioning to virtual instruction. Third, students’ perception of team satisfaction was most associated with team conflict rather than psychological safety, which may largely deal with the different expectations of the work of team assignments. Moreover, the ‘Interacting with Teammates’ dimension of peer evaluation had less influence on team project grades.

6.1. Less improvement in teamwork behavioural skills

The transition to virtual instruction was expected to provide fewer opportunities for students to improve their teamwork effectiveness due to the loss of face-to-face interaction, which hindered communication during team interactions (Krishnakumar et al. 2022). Researchers have emphasised quality communication and social interactions as key factors in virtual education, and critical for creating an inclusive virtual environment (Francescucci and Rohani 2019; O’Dea and Stern 2022), while changes brought by the pandemic can influence the team’s ability to interact and perform effectively (Wildman et al. 2021).

In our context, first-year engineering teams faced at least two issues: (1) difficulty adjusting to working with teammates on projects, and (2) additional challenges collaborating in a virtual context (Krishnakumar et al. 2022). To become a successful virtual team, members must overcome communication, collaboration, and engagement challenges (Mery 2020). We argue that students had fewer opportunities to work closely with teams due to their unfamiliarity and limitations of online collaboration tools, which negatively affected their ability to develop teamwork competency, which ultimately resulted in less improvement in teamwork behavioural skills.

6.2. Less team conflict and psychological safety

The lower levels of conflict were unexpected but aligned with findings from other scholars. Krishnakumar et al. (2022) reported that students preferred to modify their collaboration patterns to divide-and-conquer approaches so that their personal circumstances could be accommodated. In addition, Magana et al. (2022) also presented evidence that the majority of virtual teams initially would apply a compromising or accommodating strategy for collaboration, which decreased the chance of team conflict. For example, some teams chose to be less cooperative by dividing the whole task into several pieces done by each team member. Students’ adaptive strategy for coping with the emergent virtual learning would also partially explain the lower level of team conflict, lower level of interdependence, and the dramatic drop in psychological safety observed in the Spring 2020 semester. The 2-point difference in the mean psychological safety on a 7-point Likert scale measurement in both the second and third evaluation rounds is a large effect size and bears further discussion. As Ortega et al. (2010) demonstrated the positive relationship between psychological safety and learning-oriented interactions, first-year engineering students in the emergency transitioned semester were likely to engage less in their teamwork experiences.

On average, although teams were able to overcome these challenges later in the semester to achieve similar project grades to the Spring 2019 semester, students possibly paid a high cost as they worked through conditions of low psychological safety. This finding was consistent with higher levels of stress experienced by students in the Spring 2020 semester (Bono, Reil, and Hescox 2020).

6.3. Different sets of most significant factors associated with team satisfaction

The transition to a virtual learning environment facilitated a special shift in teamwork paradigm, as evidenced by the change in the measurement of team dynamics and teamwork behavioural skills. The most significant factor predicting team satisfaction was different in the two cohorts. In Spring 2019 semester, conflict was the most important factor while in Spring 2020 semester, psychological safety was found to be the most significant one. The results indicated that first-year engineering teams struggled more with managing task, process, or relationship conflicts.

Considering the second and third most important factors influencing satisfaction, such as self-ratings of the ‘Expecting Quality’ dimension in the Spring 2020 semesters, we argue that having various expectations of the teamwork quality was a major source of team conflict. In addition, a lack of interpersonal interaction in virtual context exacerbated the conflict virtual teams faced. To cope with emergent virtual learning, team structure and functionality might change, but team members might not necessarily have the same shared mental model in this regard due to the lack of face-to-face interaction (Maznevski and Chudoba 2000), or simply not having enough allocated time to develop a shared understanding (Magana et al. 2022). Student teams tended to work on team projects more individually instead of collectively, as evidenced by a reduced level of team interdependence and consistent with earlier research (Krishnakumar et al. 2022). This led to lower development of teamwork behaviour skills and eventually resulted in less satisfaction.

Furthermore, the top critical factor in predicting team project grades no longer included the peer-rated quality of Interacting with Teammates but did include the other four CATME dimensions. The results reaffirmed our finding that the virtual collaboration mode reduced effective teamwork, especially in meaningful interactions and conflict control. However, Contributing to the Team’s Work and Having Relevant KSAs consistently remained as the essential variables to explain the individual course grade. Although data on team dynamics and team-member effectiveness indicated that students had more severe teaming experiences in the emergency transition semester, the individual course grades and team project grades were slightly higher than those in the Spring 2019 semester.

7. Practical implications

Based on the findings and insights, we offer several suggestions for instructors to cope with virtual delivery of team-based courses, especially under a stress test. Even in the case of face-to-face instruction, much team interaction happens outside the watchful eye of instructors, and peer evaluation is a common and useful tool for gathering information about team dynamics that occur outside of class time. We highly recommend that instructors utilise peer evaluation to periodically monitor and gain insights into students’ team dynamics and teamwork effectiveness to help them manage the team health and conflicts. Furthermore, to promote virtual cooperation, instructors should encourage students to inform their students the effective strategies for virtual teams’ collaboration: developing shared identity and shared context (Hinds and Mortensen 2005), setting clear goals and objectives (Erez and Somech 1996), providing conflict management training (Magana et al. 2022), and allowing team members to freely communicate within teams (Swigger et al. 2012). Lastly, it is crucial for instructors to create a collaborative and open environment to facilitate students’ sense of social presence and community of practice (Krishnakumar et al. 2022; London, Douglas, and Loui 2022; Magana et al. 2022). Nonetheless, for both virtual and residential teams, active social interaction motivates students to cognitively and emotionally engage in team tasks (Magana et al. 2022; Wut and Xu 2021).

8. Limitations and future research

While this study has provided original empirical findings in student teaming experience and the effectiveness of the team-based course during the emergent transition semester as a result of the COVID-10 pandemic, the analysis is inevitably limited in scope. Notably, the results are not disaggregated by race/ethnicity and gender, which are known to be a factor in student team experiences (Beigpourian et al. 2019; Wei et al. 2020), and in the emergency transition to virtual teaching (Warfvinge et al. 2022). However, this work aimed to provide an overall comparison without exploring the impact across demographic groups, at the risk of favouring a description of the experience of majority populations. We argue that the impact of emergency remote learning across demographic groups itself merits a separate research effort due to the complexity of the intersection of race/ethnicity, gender, and international status.

As a comparative non-experimental study, we were not able to control confounding variables or mediators, which precluded the possibility of causal conclusions for our observations and holistic analysis. We hope that the findings of qualitative research regarding this transition can begin to address questions about the underlying causes. Since this study primarily focused on student teaming experiences, we acknowledged but marginalised the role of instructors in helping students navigate this transition. An extended study could explore how instructors responded to the emergency transition to virtual instruction and how they revised the curriculum to maximise the learning and teaming experience of first-year engineering teams.

Further, we were only able to collect data from a single institution with limited repeated measurement of team dynamics across semesters, which jeopardised the generalisability of this work. Other scholars who replicate the analysis of this work with other datasets might draw different conclusions. Moreover, the student experiences of transitioning from residential to virtual instruction modes in that special semester are likely distinct from the experiences of fully online teams – further study can explore whether this distinction between teams that are designed to be virtual, teams that elect to be virtual, and teams that are abruptly forced to be virtual, has any impact on the interactions and performance of that team in terms of process loss versus process gain. It would be inappropriate and inaccurate for team dynamics in virtual teams to be simply generalised from the team dynamics in traditional in-person teams because of the immense use of technology (Huber 1990). Follow-up comparative research to investigate how student teaming experience differs across purely virtual, fully residential, and mixed modes of learning environments are planned.

Lastly, studies illustrate the importance of social connectedness (Schaubroeck and Yu 2017) and team resilience (Stoverink et al. 2020) in teamwork. To prepare future teams to be more resilient and adaptive under a stress test like the emergency virtual transition during the pandemic, researchers are encouraged to investigate the mechanism of social connectedness and team resilience in face-to-face, virtual, and hybrid team contexts. While this research is not without limitations, we hope that the findings and suggestions provided in this work will inspire further in-depth research and be useful for instructors to design and develop future team-based curriculum.

9. Conclusion

This study sought to holistically contrast the team dynamics and outcomes of first-year engineering students before and during the sudden transition to virtual learning caused by the global pandemic. Using quantitative methods guided by our conceptual framework of team-member effectiveness and dynamics, we identified three major findings associated with the emergency shift to online teaching. First, students and their peers are less likely to report improvements in teamwork behaviours. Second, students reported lower degrees of interdependence, conflict, psychological safety, and satisfaction during the semester transitioning to virtual instruction. Third, team satisfaction was most associated with team conflict rather than psychological safety. Moreover, Interacting with teammates from peer rating was no longer a predictive factor of team project grades. With modifications and help from instructors, students’ individual and team scores were not significantly changed, but data demonstrated less engagement interaction, and satisfaction. These findings suggest the importance of (1) providing accessible, flexible, and accommodating instruction and tools to promote and motivate student learning; (2) creating and sustaining social presence and guiding and encouraging students through social interaction and peer collaboration; and (3) utilising protocols, such as peer evaluation, to periodically monitor student team dynamics to identify dysfunctional teams. Our analysis and results provide strong supportive evidence that those recommendations will not only help instructors better manage student teams but also equip students with healthy and positive environments to thrive in team-/project-based courses regardless of the teaching modality.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Science Foundation under grant #2227258.

Notes on contributors

Siqing Wei

Siqing Wei received B.S. and M.S. in Electrical Engineering and is a Ph.D. Candidate in the Engineering Education program at Purdue University. His research interests span on three major research topics, which are teamwork, cultural diversity, and international student experiences. As a research assistant, he investigates how the cultural diversity of team members impacts the team dynamics and outcomes, particularly for international students. He aims to help students improve intercultural competency and teamwork competency by interventions, counseling, pedagogy, and tool selection to promote DEI. In addition, he works on many research-to-practice projects to enhance educational technology usage in engineering classrooms and educational research.

Li Tan

Li Tan is an Assistant Professor of Engineering Education Systems & Design in the Polytechnic School at Arizona State University. His research interests include academic pathways, engineering student team collaboration, first-year engineering, long-term influences of K-12 education, longitudinal datasets and methods in engineering education research, and quantitative research methods.

Yiyao Zhang

Yiyao Zhang received B.S. degrees in Applied Statistics and Mathematical Statistics from Purdue University. She is a Master's student in the Applied Mathematics and Statistics program at Johns Hopkins University. As an undergraduate research assistant, her research focused on analyzing the effect of the emergency shift to virtual instruction on student team dynamics. Her current research with the Johns Hopkins School of Medicine focuses on using process mapping to optimize sepsis diagnosis and management. She is leveraging business rules management software to coordinate multi-step tasks across clinicians, and she aims to build an innovative software system to support clinical workflows with real-time task tracking and team coordination.

Matthew Ohland

Matthew Ohland is the Dale and Suzi Gallagher Professor and Associate Head of Engineering Education at Purdue University. His research includes team formation, peer evaluation, and the longitudinal study of engineering student development. With his collaborators, he has been recognized with the best paper in the Journal of Engineering Education in 2008 and 2011 and in IEEE Transactions on Education in 2011 in addition to multiple conference best paper awards. He received the 2019 Chester F. Carlson award for Innovation in Engineering Education. Dr. Ohland is a Fellow of ASEE, AAAS, and IEEE, and has served on the IEEE Education Society Board of Governors (2007-2013) and as an Associate Editor of IEEE Transactions on Education, Chair of the Educational Research and Methods division of ASEE (2009-2011), and as a Program Evaluator for ABET. Dr. Ohland was the 2002-2006 President of Tau Beta Pi.

Appendices

Appendix 1. Table of sample demographics and descriptive summaries of teamwork behavioural skills

Table A1. Sample demographics for Spring 19 and 20 semesters.

	n (Mean)	% (Standard Deviation)
Panel A: Spring 19 semester
Gender
Male	1206	71.8
Female	321	19.1
Other	12	7.1
Race
White	922	54.9
Asian	331	19.7
Hispanic	160	9.5
Declined	88	5.2
Other	32	1.9
Internationality
Domestic students	1342	79.9
International students	197	11.7
Academic Standing
Freshman	1423	84.8
Sophomore	83	4.9
Junior	30	1.8
Senior	3	0.2
GPA of First Semester	3.348	0.516
Panel B: Spring 20 semester (sample size is 1777 with 7 NA samples)
Gender
Male	1353	76.1
Female	407	22.9
Other	10	0.6
Race
White	965	54.3
Asian	445	25.0
Hispanic	171	9.6
Declined	56	3.2
Other	133	7.5
Internationality
Domestic students	1403	78.9
International students	367	20.6
Academic Standing
Freshman	1635	92.0
Sophomore	94	5.3
Junior	38	2.1
Senior	3	0.2
GPA of First Semester	3.371	0.509

Note: The sample size is 1679 with 140 samples without response (Panel A).

The sample size is 1777 with 7 samples without response (Panel B).

Table A2. Average scores of CATME peer-rating results for teamwork behavioural skills.

	PE1	PE2	PE3	PE4
Across All Dimension
Spring 19 Semester	3.98	4.04	4.05	4.10
Spring 20 Semester	3.97	3.98	4.01	4.01
Dim C
Spring 19 Semester	3.94	4.01	4.03	4.11
Spring 20 Semester	3.97	3.95	3.98	3.99
Dim I
Spring 19 Semester	3.96	3.97	4.07	4.12
Spring 20 Semester	3.98	4.00	4.01	4.00
Dim K
Spring 19 Semester	3.86	3.95	3.96	4.02
Spring 20 Semester	3.89	3.91	3.95	3.94
Dim E
Spring 19 Semester	3.98	4.06	4.08	4.11
Spring 20 Semester	3.94	3.97	4.00	4.00
Dim H
Spring 19 Semester	4.14	4.13	4.14	4.16
Spring 20 Semester	4.10	4.10	4.10	4.10

Note: PE1 to PE4 stands for the first to the last round of evaluation respectively.

Table A3. Average scores of CATME self-rating results for teamwork behavioural skills.

	PE1	PE2	PE3	PE4
Across All Dimension
Spring 19 Semester	3.94	4.00	4.05	4.11
Spring 20 Semester	3.94	3.94	3.98	4.03
Dim C
Spring 19 Semester	3.86	3.92	3.97	4.06
Spring 20 Semester	3.90	3.85	3.92	3.99
Dim I
Spring 19 Semester	3.95	4.00	4.09	4.15
Spring 20 Semester	3.96	3.98	4.02	4.04
Dim K
Spring 19 Semester	3.84	3.97	4.02	4.08
Spring 20 Semester	3.87	3.92	3.00	4.03
Dim E
Spring 19 Semester	3.99	4.02	4.10	4.14
Spring 20 Semester	3.94	3.96	4.00	4.04
Dim H
Spring 19 Semester	4.05	4.06	4.06	4.10
Spring 20 Semester	4.00	3.99	4.00	4.03

Note: PE1 to PE4 stands for the first to the last round of evaluation respectively.

Appendix 2. Instruments

Table

Team interdependence (Vegt, Emans, and Van de Vuert 2001)
Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree Nor Disagree, 4 = Agree, 5 = Strongly Agree
Items	Item Questions
TI1	My teammates and I have to obtain information and advice from one another in order to complete our work
TI2	I depend on my teammates for the completion of my work
TI3	I have a one-person job; I rarely have to check or work with others (scale reversed)
TI4	I have to work closely with my teammates to do my work properly
TI5	In order to complete our work, my teammates and I have to collaborate extensively
Team Conflict (Jehn and Mannix 2001) Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree Nor Disagree, 4 = Agree, 5 = Strongly Agree
Items	Item Questions
Task Conflict
TC1	How much conflict of ideas is there in your work group?
TC2	How frequently do you have disagreements within your work group about the task of the project you are working on?
TC3	How often do people in your work group have conflicting opinions about the project you are working on?
Relationship Conflict
RC1	How much relationship tension is there in your work group?
RC2	How often do people get angry while working in your group?
RC3	How much emotional conflict is there in your work group?
Process Conflict
PC1	How often are there disagreements about who should do what in your work group?
PC2	How much conflict is there in your group about task responsibilities?
PC3	How often do you disagree about resource allocation in your work group?
Psychological Safety (Edmondson 1999)
Scale: 1 = Very Inaccurate, 2 = Inaccurate, 3 = Slightly Inaccurate, 4 = Uncertain, 5 = Slightly Accurate, 6 = Accurate, 7 = Very Accurate
Items	Item Questions
PS1 (scale reversed)	If you make a mistake on this team, it is often held against you
PS2	Members of this team are able to bring up problems and tough issues
PS3 (scale reversed)	People on this team sometimes reject others for being different
PS4	It is safe to take a risk on this team
PS5 (scale reversed)	It is difficult to ask other members of this team for help
PS6	No one on this team would deliberately act in a way that undermines my efforts
PS7	Working with members of this team, my unique skills and talents are valued and utilised
Team Satisfaction (Vegt, Emans, and Van de Vuert 2001)
Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree Nor Disagree, 4 = Agree, 5 = Strongly Agree
Items	Item Questions
TS1	I am satisfied with my present teammates
TS2	I am pleased with the way my teammates and I work together
TS3	I am very satisfied with working in this team