Interactive technologies through the lens of team effectiveness: an interdisciplinary systematic literature review

ABSTRACT

Although interactive technologies increasingly shape teamwork, their relationship with team effectiveness (inputs, processes, emergent states, and outputs) remains unclear. To provide an overview of this relationship, we systematically reviewed empirical articles from Work and Organizational Psychology (WOP) and Human-Computer Interaction (HCI). To bring the two disciplines closer, we analysed 37 papers that validated the effects of interactive technologies, focusing on the type and characteristics of these technologies, the psychological mechanisms that they intended to support, and the methodological information of the conducted studies. We found that interactive technologies had mainly positive effects on various team effectiveness components (e.g., action team processes and task-related outputs), especially when they allowed team members to be physically close to each other or to have the option to interact synchronously. Nevertheless, the picture remains incomplete (e.g., limited evidence about affect-related properties and outputs), with several methodological limitations (e.g., mainly experimental studies with student teams). We discuss ways to shape the existing technological potential for effective team functioning, especially for affective and implicit psychological mechanisms. We highlight the need for interdisciplinary research and present an exemplary approach as an inspiration for WOP and HCI to work together and move beyond the boundaries of each single discipline.

KEYWORDS:

• Systematic review
• literature
• review
• interdisciplinary
• teams
• interactive technology

Introduction

Teams reflect an essential means, in almost all disciplines and organizations, for dealing with today’s changing and demanding conditions while performing complex tasks (J. E. Mathieu et al., 2019). To support both remote and co-located teams¹ to coordinate and communicate effectively, interactive technologies have become an integral part of the organizational setting. Interactive technologies are “methods, tools or devices that allow various entities (individuals, machines, or organizations) to engage in mediated communication to facilitate the planning and consummation of exchanges between them.” (Varadarajan et al., 2010, p. 97). In the present paper, we focus on interactive technologies whose communicating entities are team members and which vary in their degree of interactivity (Bolton & Saxena-Iyer, 2009). At the low end of interactivity, interactive technologies can include static websites, whereas at the high end of interactivity, they can include robots or AI-powered interlocutors. Such interactive technologies are increasingly being applied to assist team interactions (DeCostanza et al., 2018). For instance, video conferencing tools, such as Zoom, Skype, or Cisco Webex, are used by more than 78% of corporate companies (Czmut, 2021). This illustrates the strong integration of interactive technologies and teamwork² in today’s organizational world.

Although technology has become ubiquitous in our everyday lives, encompassing both leisure and work activities, there is a clear interest in designing and implementing technologies for teams in the work context. This is also evident in the expanding research on this topic, from both the field of Human Computer Interaction (HCI) and of Work and Organizational Psychology (WOP). On the one hand, HCI research focuses mainly on the design, evaluation and implementation of interactive technologies for teams (Grudin & Poltrock, 2012). On the other hand, WOP research mainly investigates the impact of technology use on teamwork (e.g., J. E. Mathieu et al., 2019). In other words, studies have mainly focused either on the design and development of interactive technologies for teams (HCI focus) or on the general impact of interactive technologies on teamwork (WOP focus).

Despite the increasing use of interactive technologies by teams and the empirical evidence highlighting their potential to support teamwork (e.g., X. Zhang et al., 2011), a complete picture of how interactive technologies impact the mechanisms required for effective teamwork remains largely unknown. HCI research has considered different characteristics such as modality (e.g., Chow et al., 2019) and mobility (e.g., Bellotti & Bly, 1996; Haliburton et al., 2021) when designing interactive technologies to support teamwork but has often neglected to specify the psychological mechanism that they aim to impact (e.g., specific team processes or outputs). Similarly, WOP research has investigated the impact of interactive technologies on specific psychological mechanisms in teams by treating interactive technologies mainly as uniform (Landers & Marin, 2021). It seems that the subfields of HCI and WOP with a focus on team technology research exist mainly independently from one another, investigating different aspects of the same phenomenon (i.e., developing interactive technologies while considering the different design characteristics to support teamwork in HCI; investigating the impact of interactive technologies while treating them as uniform on specific psychological mechanisms in teams in WOP). This is problematic because it does not allow for building a holistic view of how different interactive technologies and various psychological mechanisms of teams are integrated in the real world, treating their design and impact as distinct elements, instead of different components of the same phenomenon. Further, there is a lack of integration between the extensive theoretical work on team effectiveness, which demonstrates the importance of various psychological mechanisms (e.g., J. E. Mathieu et al., 2019) and the existing innovation potential and diverse characteristics of interactive technologies that exist (Fallman, 2003). To overcome these challenges and to gain a better understanding of the digital transformation and the future of work, calls for interdisciplinary approaches have become louder (Parker & Grote, 2020).

In an effort to address the above points and provide an overview of how interactive technologies shape team effectiveness to date, we review empirical work from HCI and WOP research that has validated the effect of interactive technologies in teams. Specifically, we present their current impact and future potential on various psychological mechanisms in teams. To this end, we use the team effectiveness model (Ilgen et al., 2005) as our framework and distinguish between team effectiveness components (i.e., inputs, team processes, emergent states, outputs) and specific psychological mechanisms (e.g., coordination, trust, team performance) that have been investigated when using interactive technologies in teams. Consistently, we consider the type (Pettersson et al., 2018) and time – space characteristics (Penichet et al., 2007) of the interactive technologies and review their actual impact on different psychological mechanisms (i.e., positive, negative, or no impact). Thereby, we recognize the relevance of both psychological mechanisms and the diversity of interactive technologies (Landers & Marin, 2021), moving beyond prior work that has treated either “teamwork” or “technology” in a uniform way (J. E. Mathieu et al., 2019; Mak & Kozlowski, 2019). Furthermore, we focus on all teams that use such methods, tools, or devices for communication and collaboration, shifting away from concentrating only on virtual teams defined by geographical dispersion (Kirkman & Mathieu, 2005). With our review, we aim to provide a more complete view of team effectiveness in the digital age and discuss how both HCI and WOP can complement and learn from each other.

Theoretical background

In recent decades, teamwork has been fundamentally shaped by interactive technologies. This is quite evident in the fact that technology in teams has been highlighted as an essential element that shapes team effectiveness and its components (J. E. Mathieu et al., 2019). Team effectiveness is typically understood through the input – mediator–output model (Ilgen et al., 2005), which distinguishes between inputs, mediators, and outputs. Inputs refer to the individual, team, and organizational resources, such as (team) skills or financial means, available to perform the relevant tasks (J. Mathieu et al., 2008). Mediators refer to team processes and emergent states by which resources (inputs) are translated into team results (Ilgen et al., 2005). Team processes are defined as ‘members’ interdependent acts that convert inputs to outcomes through cognitive, verbal, and behavioural activities directed towards organizing task work to achieve collective goals” (Marks et al., 2001, p. 357) and are categorized into action, transition, and interpersonal (LePine et al., 2008). Emergent states are “properties of the team that are typically dynamic in nature and vary as a function of team context, inputs, processes, and outcomes” (Marks et al., 2001, p. 357) and can be cognitive, affective, or motivational (T. Rapp et al., 2021). Lastly, outputs reflect the results of the use of the resources through the mediators, manifested in the performance, attitudes, and behaviours of teams and their members (J. Mathieu et al., 2008).

Recognizing the increasing importance of technology for team effectiveness (J. E. Mathieu et al., 2019), WOP research has mainly focused on the impact of team virtuality (e.g., Kirkman & Mathieu, 2005), the extent to which team members use interactive technologies to collaborate and communicate, and the relationships between psychological mechanisms within virtual teams (e.g., Hill et al., 2014). Specifically, many studies have investigated differences between teams who use a technology and teams who do not (Larson & DeChurch, 2020), but have often neglected the complexity of the technology itself (Landers & Marin, 2021). Although interactive technologies are neither uniform nor stable, and teams rely heavily on them, WOP researchers have mainly studied team psychological mechanisms (e.g., team processes and emergent states) within the virtual context and not the features of the interactive technology, resulting in an incomplete view. For example, although previous work has shown mixed findings about the relationship between team virtuality and team performance (e.g., Gilson et al., 2015), researchers have mainly suggested various psychological factors (e.g., work design; Handke et al., 2020) to explain these relationships and not factors related to the features of interactive technologies (e.g., synchronous or asynchronous interaction). Similarly, to explain the relationships between psychological mechanisms, such as team communication and team performance, within virtual teams, researchers have proposed other psychological factors (e.g., trust, cognition) to understand mixed findings (Marlow et al., 2017) and not factors related to the characteristics and the type of interactive technologies that virtual teams use. This is quite surprising because interactive technologies are designed and developed for different purposes, and thus their impact might differ depending on what is being investigated.

HCI research, and more specifically Computer Supported Cooperative Work (CSCW), has also focused on understanding and designing interactive technologies that support communication, collaboration, and coordination between team members, teams, or organizations (Grudin, 2017). Although this focus has remained stable over the last few decades, the interactive technologies themselves and their characteristics have changed radically (Wallace et al., 2017). For instance, today’s computers are not comparable to computers developed three decades ago. Originally, email did not support attachments and was used as a medium for informal conversations because it was too expensive to save messages over a longer period of time. As a result, a strong focus in HCI and CSCW is placed on understanding the interactive technologies themselves and how they are used, mainly by conceptualizing and applying theoretical taxonomies and typologies describing their characteristics. Such characteristics can help build an understanding of the relationship between specific types of technologies, their characteristics, and the mechanisms they intend to support. For example, one of the most established taxonomies is the so-called time – space matrix by Johansen (1988). The time – space matrix consists of two axes. The time axis is separated into synchronous and asynchronous (i.e., same time, different time). The place axis is separated into same space and different space. Up until two decades ago, interactive technologies were mostly placed in one of the cells. Today, an interactive technology can be placed in multiple cells, given that it often provides multiple different possibilities on how it can be used (e.g., synchronous and asynchronous). To capture the complexity of today’s interactive technologies, Penichet et al. (2007) extended the matrix, proposing all possible time – space combinations. This extension allows us to build a structured understanding of today’s interactive technologies and their functionalities in teams. Further, work in HCI and CSCW has also considered the type of interactive technologies when designing or implementing them. For example, one established way in HCI research to cluster technology types is the analysis of products studied in HCI and user experience research by Bargas-Avila and Hornbæk (2011) and more recently by Pettersson et al. (2018). Categories discussed in their work encompass mobile phone/app, interactive game, website, professional software, virtual reality/augmented reality, wearable, robot, amongst others, which were developed based on a qualitative analysis of more than a decade of HCI research (2005–2016).

Overall, HCI work on interactive technologies for teams is active and fruitful, but with a strong focus on the design and implementation of interactive technologies themselves and on interactions between humans and technologies as well as teamwork in general. Many studies on HCI have focused on designing and developing technologies that support teams in extending their capabilities (Hornbæk & Oulasvirta, 2017; K. Schmidt & Bannon, 2013). For instance, if a team member uses augmented reality that visualizes instructions on how to execute a specific task, this team member extends its capabilities by using that interactive technology (e.g., Funk et al., 2017). To achieve this goal and to develop a deep understanding of a team’s experience in using such an interactive technology, studies have explored how teams and their members interact with a specific technology, and how the technology, in turn, can be adjusted to the team’s needs. Related research has also investigated how the use of a specific interactive technology can predict team mechanisms. For instance, Cao et al. (2021) analysed the chat interactions between team members and trained machine learning models to predict the general viability of virtual teams. Given the increase in remote work, HCI studies have also focused on the development of interactive technologies to support virtual teamwork. For instance, Müller et al. (2017) explored mixed reality, the merging of real and virtual words, and established a concept of shared virtual landmarks to ensure common reference points for remote members. Similarly, McGregor and Tang (2017) developed a technology that automatically generates “action items” to support team collaboration based on meeting transcripts.

We argue that to understand relationships and outcomes when using interactive technologies in the team context, we need to treat the range and multitude of facets of interactive technologies and psychological mechanisms as equally important. As Demerouti (2020) recently highlighted, we should aim for a “human and technology symbiosis” (p. 2). Hence, there is a need to build a more integrated understanding of the interplay between the characteristics of interactive technologies and psychological mechanisms in teams. This is also in line with related theoretical work in WOP, highlighting that to understand the impact of technology on outputs, we need to align the features of the technology with the given demand or task that the team is facing. According to the task – technology fit paradigm (D. Goodhue, 1988; D. L. Goodhue & Thompson, 1995), matching the appropriate technological features to the demands imposed by a given task should result in better outcomes. In line with this paradigm, Maruping and Agarwal (2004) showed that teams executed their interpersonal processes more effectively and reached higher outcomes when there was a greater alignment between the communication requirements of the task and the specific features of the technology.

Building on the above work from both WOP and HCI research, we propose that to gain an overview of how interactive technologies indeed shape the psychological mechanisms in teams and thus team effectiveness, we need to consider the type of interactive technology and its characteristics, the psychological mechanisms that it is designed to support, and whether it meets its purpose (i.e., actual impact). For this purpose, we follow an interdisciplinary paradigm, as Landers and Marin (2021) recently proposed, and review empirical studies from both WOP and HCI that have explored the influence of interactive technologies on psychological mechanisms in teams. Specifically, we consider the time – space characteristics of interactive technologies as extended by Penichet et al. (2007) and the types of interactive technologies proposed by Pettersson et al. (2018). We also apply the lens of the team effectiveness model (Ilgen et al., 2005) and use the input – mediator–output (IMOI) logic as an organizing heuristic for psychological mechanisms in teams. To date, this IMOI heuristic and the team effectiveness model have been successfully applied to review the literature related to team effectiveness (J. Mathieu et al., 2008) and to other topics, such as team adaptation (Maynard et al., 2015) and empowerment in teams (Maynard et al., 2012).

Method

To shed light on the diverse research landscape of interactive technologies for teams and how they impact team effectiveness, we adopted a systematic approach. A systematic review allows one to address the research objective by identifying, critically evaluating, synthesizing, and integrating the findings of relevant research (Cooper, 2003). This methodology enabled us to identify the current state of interactive technologies for teams in both WOP and HCI research, to synthesize the two fields in a structured way, and to provide a future roadmap for enriching both fields. Our focus was to identify work that has investigated the impact of such interactive technologies on psychological mechanisms in teams, and that has not only focused on their design or development without validating them. This allowed us to systematically capture the type and characteristics of the interactive technology (Penichet et al., 2007; Pettersson et al., 2018), the psychological mechanisms that it was intended to influence (Ilgen et al., 2005), and the actual impact of the interactive technology on these psychological mechanisms (i.e., positive, negative, and no impact). In line with Briner and Denyer (2012), we designed this systematic review to specifically address our research goal, describing in detail all the steps followed so that the review could be replicated. An overview of our procedure, including all the steps, is shown in Figure 1.

Figure 1. Flow Diagram of steps included in the identification and screening coding process.

Identification of interactive technologies

To explore the state of the art of interactive technologies for teams encompassing both WOP and HCI research, we used two databases, APA PsycInfo and the Association for Computing Machinery Digital Library (ACM DL). The APA PsycINFO database contains peer-reviewed academic literature on WOP and related disciplines. The ACM is the world’s largest scientific and educational computing society. ACM DL is the most comprehensive database in the academic computing and information technology literature. It includes a complete collection of ACM publications (both peer-reviewed and juried) and an extended bibliographic database of core works in computing from scholarly publishers.

The following terms were used for the database searches: (1) “team(s)” and (2) “tool(s)” or “technology/technologies”, always combining the first term with one of the other two. We used the term “team” due to our focus on interactive technologies developed for teams instead of single users. We used both “tool” and “technology” as terms to ensure that a broad variety of systems was included, given that both terms are often used interchangeably in the WOP and HCI literature. The search was performed on the title, abstract, keyword, and topic and focused on publications in the English language. In PsycInfo, the search was restricted to peer-reviewed journal articles. In the ACM DL, the search included journal articles and full conference papers. Both types of publication undergo a peer-review process. The publication model in the field of HCI is mainly based on conferences, meaning that conferences are often as prominent or more prominent than journal publications. Research articles published at relevant HCI conferences undergo a double-blind peer-review process with a total of at least three external reviewers per paper, with acceptance rates of 30% or lower.

There was no search start date, and the end date was July 30^th, 2020. Key information on all the articles (e.g., year of publication, authors, title, abstract), without further limits (e.g., up to which page it was searched), was downloaded and compiled into a Microsoft Excel spreadsheet. In total, 8100 peer-reviewed articles (PsycInfo: 4544, ACM DL: 3556) were identified.

Screening process

The screening process included two steps. First, the title and abstract of each article were read by three coders (i.e., research assistants). The coders used seven inclusion and exclusion criteria.: First, (1) all duplicates were removed. Then, due to our particular focus on interactive technologies designed for and validated with working teams, all studies that (2) did not contain empirical data (e.g., literature review); (3) focused on an interactive technology that was not designed for working teams (e.g., pupil teams); (4) presented the validation of a measurement scale (e.g., technology acceptance scale) instead of an interactive technology; (5) validated an interactive technology with participants outside of the team (e.g., used by patients of medical teams); (6) and investigated the impact of multiple interactive technologies at once³ were excluded. Finally, as the HCI research landscape publishes mainly at conferences and includes a variety of specialized conferences that might not yield results relevant to our research aim, (7) we used a Microsoft Academic search to obtain conference and journal rankings in HCI. We included 12 of the most relevant conferences out of the top 20 conferences (for details, see Appendix A) based on an expert discussion that reported that these conferences publish work related to our topic (3 researchers involved). In cases where one of the coders⁴ was unsure if a paper should be excluded, the paper was marked and screened in detail in the second step. In total, 394 articles remained (PsycInfo: 360, ACM DL: 34).

Second, the 394 articles were downloaded, read in full, and screened by three coders (i.e., research assistants) according to the same seven criteria described above. In cases where one of the coders was unsure if a paper should be excluded, the paper was marked and screened by one of the authors. This second step led to a final set of 37 articles (PsycInfo: 25, ACM DL: 12). As some articles reported multiple studies to validate one or more interactive technologies, our overall screening process resulted in 37 articles, 35 interactive technologies, and 38 studies.

Coding process

After the screening process, two of the authors-one experienced in WOP team research and one experienced in HCI research – developed a coding scheme. The main aim of the coding scheme was to ensure that the type and characteristics of the interactive technologies, the psychological mechanisms, and the actual impact of the interactive technologies on these psychological mechanisms were captured. Furthermore, other methodological information (e.g., research design; for details, see content analysis), relevant for both WOP and HCI research, were extracted. The coding of studies was as detailed as possible to provide a comprehensive review of the existing interactive technologies for teams and how they currently shape team effectiveness.

To achieve our goal, we coded based on three main criteria: (1) the interactive technology itself and its characteristics; (2) the psychological mechanisms investigated; and (3) the impact of the interactive technology on the respective psychological mechanism(s). Each of the three criteria included various coding categories (for details, see below). Each category included in the coding scheme was discussed and clearly defined to create a shared understanding across WOP and HCI research. Clear definitions, which were based on previous theoretical and empirical work from one or both research streams, ensured that the same underlying information was consistently coded across disciplines. In addition to these three criteria and their categories, we also coded information about the research design adopted in the studies to gain, from a methodological point of view, a better understanding of the current state of research.

After the coding scheme was finalized, the two authors and one subject-matter expert, who was not included in the previous steps and had experience in teams and interactive technologies, independently coded an initial set of five articles. After this initial coding, a meeting was held in which all three coders came to a consensus regarding any discrepancies in their codes and accordingly made some minor revisions (e.g., revised definitions, included additional categories). The coding scheme was then finalized (see Appendix A). In the final step, each of the three coders coded a third of the remaining articles using the final coding scheme. In the case of uncertainty, the articles were coded by one more coder. In the case of discrepancies, the coders discussed the articles until a consensus was reached.

Content analysis

Interactive technology

With regard to the interactive technology itself and its characteristics, we coded the 35 interactive technologies using the following coding categories:

Time – space Characteristics

We coded the characteristics of the interactive technologies, as described in the articles, according to time and space, in line with the original work by Johansen (1988) and the non-exclusive classification proposed by Penichet et al. (2007). Specifically, we used nine time – space characteristics, with interactive technologies being synchronous, asynchronous, or both, and with team member interaction taking place in the same physical space, in a different one, or in both.

Type of Technology

We coded the type of interactive technology using the categories discussed by Pettersson et al. (2018), which were in line with our definition of interactive technologies. The categories were mobile phone/app, interactive game, website, professional software, virtual reality/augmented reality, wearable, robot, and other.

Psychological mechanisms

With regard to the psychological mechanisms investigated, we adopted the team effectiveness framework by Ilgen et al. (2005) and coded different team effectiveness components, specifically inputs, mediators (team processes and emergent states), and outputs, as well as the specific psychological mechanisms (for reviews, see J. E. Mathieu et al., 2019; J. Mathieu et al., 2008) explored in the 38 studies.

Inputs

We coded individual-level (attributes of team members), team-level (the combination of such attributes or team characteristics), or organizational-level inputs (external factors defining the larger organizational system within which the team belongs). Categories (i.e., specific psychological mechanisms) for inputs at the individual level were personality, competencies, goal orientation, teamwork orientation, and personal values (J. Mathieu et al., 2008). The categories for inputs at the team level were interdependence, team virtuality, team training, team coaching, team leadership, and cultural diversity in the team (J. Mathieu et al., 2008). The categories for inputs at the organizational level were human resource system, openness climate, cultural diversity in the organization, and organizational culture (J. Mathieu et al., 2008).

Mediators – Team Processes

We coded transition, action, or interpersonal team processes (Marks et al., 2001). The categories for transition processes were mission analysis, goal specification, strategy formulation, and planning (Marks et al., 2001), as well as team reflection and team learning (e.g., Georganta et al., 2021). The categories for action processes were monitoring progress towards goals, systems monitoring, team monitoring and backup, coordination (Marks et al., 2001), and the process of task execution (e.g., Aggarwal & Woolley, 2013). The categories for interpersonal processes were conflict management, motivation and confidence building, and affect management (Marks et al., 2001). Lastly, we coded the process of team communication (verbal and nonverbal; Marlow et al., 2018).

Mediators – Emergent States

We coded the cognitive, motivational, and affective states of teams (Marks et al., 2001). The categories for cognitive emergent states were transactive memory systems and shared mental models (DeChurch & Mesmer-Magnus, 2010). The categories for motivational states were team efficacy, team potency, team empowerment, and team motivation (J. Mathieu et al., 2008). The categories for affective emergent states were trust, cohesion, and climate (J. Mathieu et al., 2008).

Outputs

We coded outputs related to performance, attitudes, and team-level behaviours (J. Mathieu et al., 2008). The categories for outputs were team performance, time for task completion, group member satisfaction, creativity, and innovation.

Impact of interactive technology

With regard to the impact of interactive technology on the investigated psychological mechanism(s), we coded whether the effect was positive or negative. When the impact was not significant, the category we used was non-significant.

Research design

With regard to research design, we coded 38 studies. Specifically, we coded the type of publication (i.e., journal article or full conference paper), the study design (i.e., true experimental design, quasi-experimental design, correlational design, interview study, focus group, workshop, and mixed design), whether the study was cross-sectional or longitudinal, and the measures used to assess the psychological mechanisms (e.g., scale), and the source from which the psychological mechanisms were captured (i.e., self, team members rating one another, observer/independent rater, supervisor, subordinate, automated/objective, team consensus, customer/client, stakeholder, and mix of sources). Furthermore, we coded the type of sample (i.e., college students, employed adults, top management teams, and mixed), the sample’s region (North America, Asia, Australia, Latin America, Middle East, South America, India, Israel, mixed) and country, the number of teams completing the study, team distribution (i.e., distributed, partially distributed, and collocated), team stability (i.e., ad-hoc, pre-existing intact, or newly formed intact), and team membership (i.e., hours the teams worked together for the purposes of the study).

Interactive technologies and team effectiveness

Our review is divided into four main sections. First, we begin by reviewing⁵ how interactive technologies impact inputs. Second, we review studies that have examined how interactive technologies impact team processes as mediators. Third, we review the impact of interactive technologies on emergent states as mediators. Fourth, we review how interactive technologies impact outputs. Within each section, we highlight some key work and primarily focus on understanding how interactive technologies currently shape team effectiveness and where future opportunities lie (Table 1, Appendix B). Specifically, we focus on the type of impact (negative, positive, non-significant) from the different interactive technologies on the respective psychological mechanisms (Figure 2). Furthermore, we also consider the type and time-space characteristics of the interactive technologies when discussing their relationship with psychological mechanisms (Figure 3) as well as the research design adopted.

Figure 2. Illustration of 81 investigated relationships (positive, negative, no impact) between different interactive technologies (robot, website, software, mobile phone/app, VR/AR, interactive games, IOT) and psychological mechanisms across 35 papers.

Figure 3. Classification of the 81 investigated relationships (positive, negative, non-significant) according to the time-space characteristics of the different types of interactive technologies (robot, website, software, mobile phone/app, VR/AR, interactive games, IOT).

For an overview of the main extracted information of the 37 accepted papers, 38 studies, and 35 interactive technologies, see Table 2 (for a complete overview, seeAppendix C).

Impact of interactive technologies on inputs

Inputs of team effectiveness, compared to other psychological mechanisms, have not yet received a great deal of consideration when investigating the impact of interactive technologies. Nevertheless, the majority of studies focused on the relationship between interactive technologies and inputs and found the positive effects of professional software. With regard to the time – space characteristics of these interactive technologies, team interaction was either synchronous or asynchronous and took place in different physical spaces, with most of the studies investigating sufficient sample sizes consisting of student teams that worked together for less than a month.

In terms of individual-level inputs, studies mainly examined individual competencies and teamwork orientation using longitudinal and experimental study designs and found that both were supported by interactive technologies. For instance, T. L. Rapp and Mathieu (2007) showed that their professional software, an individually delivered, technology-based (CD) training programme, had a positive effect on team members’ knowledge skills about teamwork. Relatedly, Heidrich et al. (2015) demonstrated that their websites, modern Web 2.0-based collaborative technologies (wikis), positively supported various team member competencies, such as the ability to share knowledge and the ability to learn from mistakes. However, one longitudinal field study with student teams (Blau et al., 2019) found a negative relationship between the individual-level input of personal values and their asynchronous interactive technology, the mobile app DevelapMe. Similarly, Glikson et al. (2019) found no impact of their visualized automatic feedback professional software on the team composition of student teams. In terms of team-level inputs, studies showed that the professional software LiveDeck (Scissors et al., 2011), which was the only interactive technology studied with employed adults and intact-old teams, and the LCC Collaborative Activity professional software (Cortez et al., 2009), both asynchronous interactive technologies, had a positive impact on the interdependence between team members and team leadership in terms of identifying the most influential team member for team decisions and coordination. In terms of organizational-level inputs, we found no study that investigated how they are influenced by interactive technologies. Taken together, the impact of interactive technologies on inputs has not received considerable attention, with studies taking place mainly under controlled conditions and interactive technologies being validated largely with student samples. Nevertheless, professional software and websites seem to support individual- and team-level inputs when team interaction in different physical spaces is possible. Future research should pay more attention to the relationship between interactive technologies and inputs by adopting research designs that are closer to the organizational setting and real working teams.

Impact of interactive technologies on team processes

Team processes as mediators of team effectiveness have been examined considerably in relation to interactive technologies in recent years. Although the impact of different types of interactive technologies (i.e., professional software, website, app, robot, virtual reality) has been investigated, the majority of studies have focused on professional software and demonstrated positive effects. With regard to the time – space characteristics of the interactive technologies, effects were positive when team interaction was mainly synchronous and took place in the same or different physical space. When considering the research design adopted, the studies covered various study characteristics and team types, with half being cross-sectional and half being longitudinal. Still, the sample size was relatively small, with fewer than 15 teams participating in more than half of the studies. Student teams were mainly used; however, team processes were the only team effectiveness component that was also reasonably investigated with working teams. The majority of teams worked for some months together and were investigated largely using an experimental design, followed by a mixed design.

In terms of transition processes, studies have examined mission analysis, strategy formulation, and planning, and particularly team reflection and team learning, showing positive relationships with interactive technologies. For instance, in a longitudinal study with intact new teams consisting of teachers, Gross et al. (2001) showed that their website qesn.connection, a tool for professional development with multiple features that allows synchronous interaction in the same physical space, supported team members in giving feedback to each other about their teaching activities to allow learning and improvement. Similarly, in Blau et al. (2019) study, ad-hoc student teams reported being more open in receiving positive and negative feedback from their team members when using the mobile app DevelapMe, which allows asynchronous interaction in different physical spaces.

Although some studies focused on transition processes, the majority of research investigated the influence of different types of technologies on action processes, namely monitoring progress towards goals, team monitoring and backup, and especially coordination and team communication, demonstrating largely positive effects. For instance, in Montoya et al. (2011) experimental study, teams reported that team coordination improved after using virtual reality, namely, the 3D collaborative virtual environment Linden Lab‘s Second Life platform. Vyas et al. (2016) also observed that student teams improved their coordination when they used the internet-of-things technology called CAM. However, Fu et al. (2018) found that the coordination of a small student sample declined when it used the professional software T-Cal, which visualizes conversational data. Nevertheless, team communication improved when they used the same technology. Similarly, Lin et al. (2016) qualitative study demonstrated that team communication of distributed ad-hoc working teams improved with use of the website Slack, which allows synchronous team interaction from different physical spaces. Qureshi et al. (2006) also showed in their longitudinal study that distributed student teams improved their team communication when they used the professional software eRoom, which allowed synchronous team interactions from different spaces. However, Neff et al. (2010) case study showed that team communication of working teams decreased when they used the professional software BIM.

In addition to the studies on action or transition processes, some work has also investigated the impact of interactive technologies on various team processes together, without differentiating among them. For instance, T. L. Rapp and Mathieu (2007) experimental study found that student teams improved their transition, action, and interpersonal processes when using the professional software CD-based Teamwork Training, an individually delivered, technology-based training programme. Overall, there were mainly positive effects of interactive technologies on team processes, and rich evidence with regard to the type of interactive technologies investigated and the research designs adopted. However, the sample size of many studies was small, an important methodological limitation that should be considered in future research, along with the fact that no studies have so far explored the relationship between interactive technologies and interpersonal processes.

Impact of interactive technologies on emergent states

Emergent states as mediators of team effectiveness have so far received little attention with regard to their relationship with interactive technologies. Although some studies have found positive effects, especially when investigating professional software that allows synchronous team interaction in the same space, others that have focused on the impact of the robot as an interactive technology have found negative relationships. With regard to research design, most studies have adopted an experimental design and investigated ad-hoc student teams, who worked together for a short period of time.

In terms of cognitive states, studies have investigated transactive memory systems and shared mental models, demonstrating mixed effects. For instance, on the one hand, Stone et al. (2017) experimental studies showed that student teams improved their shared mental models in terms of team understanding when using the professional software Telestrator, a basic collaborative sketching application. On the other hand, Gumienny et al. (2014) found no impact of the professional software Synthesis Guide on shared mental models, and Lazzara et al. (2015) showed that RP-7, a telemedical robot on trauma intensive care units, had a negative impact on the transactive memory systems of real medical teams. Lazzara et al. (2015) study, which was one of the few studies with working teams as participants, also found a negative relationship between using the RP-7 robot and two affective emergent states, namely trust and climate. In contrast to these findings, in a longitudinal survey study, Blau et al. (2019) showed that the mobile app DevelapMe supported the trust of student teams. Ardaiz-Villanueva et al. (2011) similarly showed that the climate of student teams improved when using the Wikideas and Creativity Connector websites, which include web-based interactive tools for idea generation and evaluation. In terms of other affective states, a few studies have also investigated cohesion, demonstrating positive effects. For instance, Oren and Gilbert (2012) found that the professional software ConvoCons, a technology for team members working in a co-located environment, improved the observed cohesion of real working teams with respect to liking each other. With regard to motivational emergent states, we found no study that investigated how they are related to interactive technologies. Taken together, the impact of interactive technologies on emergent states is currently mixed and insufficiently understood. To gain a more complete picture, future work should move beyond student samples under controlled conditions and cover a larger spectrum of interactive technologies and emergent states.

Impact of interactive technologies on outputs

Team outputs of team effectiveness have received a great deal of consideration in investigating the impact of interactive technologies. Specifically, studies have explored the relationship between different types of interactive technologies (i.e., mobile app, interactive game, website, professional software, internet of things, virtual reality, and robot) and outputs, demonstrating primarily positive effects. The relationships were positive, especially when interactive technologies allowed synchronous or asynchronous team interaction from different places, with most of the studies focusing on professional software. In terms of research design, the studies were primarily experimental with relatively small samples of ad-hoc student teams, which worked for some hours together.

In terms of the team outputs investigated, the majority of studies have focused on team performance, followed by time, group member satisfaction, and creativity and innovation. For the outputs of team performance and creativity and innovation, the relationship with interactive technologies was mainly positive. For instance, Yuan et al. (2016) found that student teams improved their team performance in terms of how many key points were identified during a problem analysis discussion when using the professional software Mutual Awareness Tool. Similarly, in the experimental study by Ardaiz-Villanueva et al. (2011), creativity and innovation, with regard to the number of ideas generated, improved when student teams used the Wikideas and Creativity Connector websites. For output time and group member satisfaction, the effects were mixed. For instance, Alberola et al. (2016) found that when using the professional software Artificial Intelligence Tools, which combines artificial intelligence techniques to facilitate the generation of working groups, student teams improved their team satisfaction. On the contrary, Jang et al. (2002) showed that team satisfaction decreased when student teams used the TeamSCOPE website, a web-based collaborative system. Similarly, in their experimental study, Yuan et al. (2016) found that student teams needed less time to make a diagnosis when using the professional software Mutual Awareness Tool. By contrast, Wu et al. (2013) showed that teams consisting of students and university employees did not work faster when using the professional software GeoTools, a collaborative sense-making technology for teamwork. Overall, studies have largely focused on outcomes, especially on team performance, when investigating the impact of interactive technologies, and have shown primarily positive effects. To move forward, we propose that future research pay more attention to group-related outcomes, such as group member satisfaction, and investigate a broader spectrum of sample types outside the experimental setting.

Discussion

The goal of our review was to provide an overview of how interactive technologies impact psychological mechanisms in teams and thereby team effectiveness, moving beyond the boundaries of a single discipline, as repeatedly proposed previously (e.g., Landers & Marin, 2021). To that end, we systematically reviewed papers from WOP and HCI that have investigated the effects of interactive technologies in teams, considering both their characteristics (Penichet et al., 2007) and the specific team effectiveness components that they impacted (Ilgen et al., 2005). We showed that different interactive technologies can have different impact on various psychological mechanisms. With our work, we highlighted the need to move away from focusing either on the extent to which team members use interactive technologies to collaborate and communicate (main focus of WOP so far) or on the effects of interactive technologies on teamwork in general (main focus of HCI so far). Overall, we contributed to a better understanding of which interactive technologies are most suitable for which psychological mechanisms in teams and where the potential lies for future work.

With regards to the time – space characteristics of the interactive technologies, psychological mechanisms were supported, especially when synchronous or asynchronous interaction in the same space was possible, or when both synchronous and asynchronous interaction in different spaces was possible. In other words, it seems that interactive technologies promote psychological mechanisms, either when team members are physically close to each other or when team members have the option to interact synchronously when they are far from each other. Synchronous interaction, similar to being physically close to each other, is related to more perceived connection, social presence, and common ground, often leading to a high degree of engagement and identification (Burgoon et al., 2010). In terms of the type of interactive technology, the impact of professional software on psychological mechanisms was investigated the most, demonstrating mainly positive effects. In contrast, robots showed mixed effects on psychological mechanisms. These mixed findings might be related to the fact that robots are often perceived differently than other types of interactive technologies. Due to their physical appearance, humans tend to humanize robots in ways that make individuals interact with them as if they were humans (e.g., Robert, 2017). This humanization of the technology may alter the way team members use robots and, thus, how robots impact the way teams operate. Our findings also revealed that the rapid progress of technological development was not mirrored by the interactive technologies investigated in relation to psychological mechanisms in teams. Despite the positive effects of more recent technological advancements (e.g., virtual reality, interactive games) on team effectiveness components, these findings were limited.

In terms of psychological mechanisms, interactive technologies mainly supported action team processes and team-related outcomes. It is possible that interactive technologies might be more suitable for promoting explicit task-related actions and results, as they are developed and improved via explicit feedback and interaction. Another possible explanation is that because these psychological factors can be captured and evaluated more directly, they were mainly investigated in relation to interactive technologies. This aspect is also related to our findings showing that transition and interpersonal processes – focusing on planning and attitudinal behaviours – have rarely been investigated. Hence, strong evidence demonstrating the relevance of all types of team processes to team success (J. E. Mathieu et al., 2019, 2020; LePine et al., 2008) has been largely neglected. Furthermore, although emergent states reflect one of the most critical drivers for reaching desirable team outcomes (T. Rapp et al., 2021), the impact of interactive technologies on this team effectiveness component has been mixed and not largely investigated. Especially regarding properties related to motivation and affect, the evidence was limited. Relatedly, there are only a few findings about the relationship between interactive technologies and affect-related outcomes, such as group satisfaction. This is surprising, given that interactive technologies have been used in other contexts to support affect-related factors, such as motivation (G. B. Schmidt, 2015) and trust between single individuals (Depping et al., 2016). Similarly, research on teams has illustrated the importance of affect for team interaction and functioning (Knight & Eisenkraft, 2015; Spoor & Kelly, 2004). Moreover, we found limited evidence regarding the impact of interactive technologies on inputs. For instance, interactive technologies supporting team training interventions (e.g., Hughes et al., 2016; Salas et al., 2008) or team virtuality (J. E. Mathieu et al., 2019; M. I. Brown et al., 2020) were missing, although team inputs gained increasing attention in shaping how teams operate and perform.

Overall, our findings revealed that different interactive technologies and characteristics have mainly positive effects on various psychological mechanisms in teams. Interactive technologies seem to generally promote team effectiveness components, an important finding given that the digital age is increasingly shaping the way teams and their members operate (Larson & DeChurch, 2020). Nevertheless, team effectiveness has not been fully captured in relation to interactive technologies, nor have studies investigated these relationships with different methodologies (e.g., laboratory and field studies, student and organizational teams). In the following sections, we discuss future directions that our review implies for empirically reaching a more complete picture of the impact of different interactive technologies on team effectiveness.

The role of affect

Affective and motivational team properties, such as team trust, team cohesion, and team confidence, are crucial in building strong interpersonal relationships and reaching high team outcomes (T. Rapp et al., 2021). For instance, trust has been found to support team performance across a broad range of team types and contexts (for meta-analysis, see De Jong et al., 2016), as well as interaction between human and intelligent systems (Schaefer et al., 2016; Sheridan, 2019). Further, interactive technologies already exist for dyads and non-working teams that can foster such affective and motivational mechanisms. For instance, “Labyrinth”, a networked, cooperative two-player online game, supported interpersonal trust among two individuals (Depping et al., 2016), while features of the augmented reality game “Ingress” improved the perceptions of single players with regard to team commitment (Morschheuser et al., 2017). Other interactive technologies, such as “Chick Clique”, a mobile app for a group of up to four friends to track walking, have been developed to promote collective motivation (Toscos et al., 2006). Recently, chatbot designs have also been positively evaluated for developing interactive technologies for emotion management and team affect awareness (Benke et al., 2020). Other work also suggests the use of interactive technologies (e.g., mobile apps) during walking team meetings to support interpersonal relationships and properties (Haliburton et al., 2021).

To move forward, we suggest that future research use the existing technological potential as a foundation to develop new interactive technologies that explicitly support affective and motivational team properties. Such technologies should be situated within the organizational setting, and their impact should be investigated with working teams. To that end, existing technologies that facilitate such properties in different contexts could be used as inspirations for future designs. This will allow us to gain a better understanding of how working teams experience and perceive the use of existing technologies (Hassenzahl et al., 2010) and whether these should be adjusted – either in their development or in their implementation – to team and organizational characteristics, such as task interdependence (T. Rapp et al., 2021).

The role of implicit mechanisms

Team members need to coordinate their actions not only explicitly (e.g., communicating the following steps to the team members) but also implicitly (e.g., anticipating the future needs of team members) to successfully operate (Rico et al., 2019). Shared cognitive structures and an understanding of who knows what is also required to benefit from everyone’s knowledge, remain flexible, and reach successful outcomes (Zhou & Pazos, 2020). As research has demonstrated, such implicit mechanisms reflect an essential component of team effectiveness across various organizational settings (e.g., Valentine & Edmondson, 2015; Weick & Roberts, 1993).

Various interactive technologies have been developed to support implicit mechanisms during interactions. For instance, K-net, a web-based social matching system, has been created to match people with specific skills to those who need them, thus fostering awareness of knowledge distribution (Shami et al., 2007). Another example is RAMPARTS, a spatially aware system supporting crime analysis that has been developed to create a shared understanding among individuals based on collected cues (Wozniak et al., 2016). Augmented reality has also shown the potential to foster implicit factors. Draxler et al. (2020) recently developed an augmented reality app that includes real-life objects in the surroundings to help users learn case grammar. Along these lines, recent work has also highlighted the potential of multimodal functions in intelligent technologies to promote factors such as non-verbal communication and implicit information exchange (Liang et al., 2019; R. Zhang et al., 2021).

We suggest that future work utilize the existing technological potential and explore whether and how this can foster implicit psychological mechanisms (e.g., transactive memory systems) of working teams. We propose building on existing work, which has been mainly completed outside the organizational setting, and examining how this can be translated to support mechanisms, such as implicit coordination, which are critical for long-term team effectiveness (Aggarwal et al., 2019).

An interdisciplinary outset

In the present review, none of the included studies applied an interdisciplinary approach to designing interactive technology and investigating its impact on psychological mechanisms in teams. This is surprising given the interdisciplinary nature of the topic itself and the fact that the different perspectives (i.e., focus on design and development of interactive technologies for teams in HCI; focus on the general impact of interactive technologies on teamwork in WOP), theoretical foundations, and methodological strengths that each discipline brings could elegantly complement each other. To consider teams and interactive technologies as more of a symbiosis (Waterson et al., 2015), we highlight, in line with previous calls, the need for interdisciplinary research. Working together will enable researchers to adopt a human-centred approach regarding the design, development, and validation of new technologies (Parker & Grote, 2020) and to consider interactive technologies more at the beginning of investigating psychological mechanisms instead of later on (Landers & Marin, 2021). To start moving towards such an interdisciplinary direction, we present a process that can act as inspiration for WOP and HCI researchers to work together.

We build on the suggestion by Landers and Marin (2021) and propose integrating the systematic process of designing and developing an interactive technology with the empirical investigation of interactive technologies in teams, incorporating the theoretical stance and methodological strengths from both HCI and WOP research. In particular, we suggest that researchers combine a human-centred process that enables a deep understanding of how the user (e.g., team) perceives and interacts with the respective environment (Parker & Grote, 2020) with existing theory and evidence while ensuring that various methodological criteria are met (e.g., generalizable target group, sufficient sample size, satisfactory psychometric properties of measures). As an example, we use the systematic process proposed by T. Brown and Wyatt (2010; for an overview, see Dourish, 2021) and present below how the different stages—(a) empathize, (b) define, (c) ideate, (d) prototype, (e) test, and (f) implement – can be executed in an interdisciplinary way from designing a new interactive technology for teams to validating it.

Starting with the (a) empathize stage, researchers should build an understanding of the psychological aspects of interest (e.g., difficulties in building team trust due to regular membership changes) and contextual challenges based on extensive data collection. Data should be collected using appropriate methods. For instance, qualitative methods, such as focus groups and ethnography, could be used to build an in-depth understanding of user needs, and quantitative methods could be used to derive insights about the broader organizational context. During this stage, methods should be shaped according to existing psychological evidence and theories (e.g., research on team trust). To ensure that unexplored aspects are identified, a balance between qualitative focus and theoretical underpinning is required. In the (b) define stage, the goal is to analyse the collected data and identify the specific team-psychological challenges (e.g., difficulties in evaluating a new team member’s abilities and skills) that should be addressed by the interactive technology. Here, the methodological strengths of both disciplines, such as systematic (e.g., content analysis) and relational (affinity diagramming, concept mapping) analyses, should be used. In the (c) ideate stage, researchers should generate different ideas that provide a possible solution for the identified problem. Thinking out of the box and brainstorming creative ideas should be encouraged to profit from the different perspectives that WOP and HCI researchers bring, considering both the innovation potential and how a team operates and functions. During the (d) prototype stage, researchers should create an understanding of which elements of the idea can be translated into an inexpensive prototypical solution. Here, the expertise in conceptualizing and designing the prototype can be complemented with existing evidence that demonstrates elements of the idea implementation (e.g., present new team members in a way that they are perceived as similar to the rest of the team) that are likely to work in the team context. In the (e) test stage, the prototype is evaluated using user feedback. Here, a combined methodological approach is suggested. For instance, in addition to focus groups, which allow the collection of detailed feedback about the user’s experience with the interactive technology, other valid and reliable measures, such as self-report scales, are proposed to be implemented pre- and post-prototype use to analyse how the interactive technology impacts the psychological aspect of interest (e.g., trustworthiness of new team members and team trust). The (f) implement stage involves the actual use of the interactive technology by end users. Here, the goal is to implement the vision in a complete product. However, one should aim for continuous improvement of interactive technology based on constant data collection. To do so, researchers can, for example, conduct longitudinal studies using a sufficient team sample size, in which the psychological mechanisms of interest are measured via reliable and valid measures at multiple points in time. Insights can also be gathered through in-depth interviews or focus groups. During the systematic process, whenever issues arise, moving to earlier stages to solve them is suggested (Bjögvinsson et al., 2012).

We believe that the above interdisciplinary suggestions can serve as possible steps to design scalable interactive technologies applicable to more than one team context and to investigate their relations to various psychological mechanisms in teams. This would allow for a more complete picture of the impact of interactive technologies on team effectiveness and, thus, of the organizational context in the digital age.

Conclusion

To gain a better understanding of the impact of interactive technologies on team effectiveness, we systematically reviewed 37 articles from the WOP and HCI research. Bringing the two disciplines closer, our systematic review considered the type of interactive technologies and their characteristics, as well as the psychological mechanisms in teams. Our findings revealed that different interactive technologies and characteristics have mainly positive effects on various team effectiveness components in teams. However, team effectiveness has not been fully investigated in relation to interactive technologies, with empirical studies having similar methodological limitations (e.g., mainly experimental studies with student teams). To move forward, we discussed how existing technological potential can be shaped into new interactive technologies, especially for fostering affect and implicit mechanisms in teams that have not received enough attention to date. Furthermore, we proposed more interdisciplinary research by describing an exemplary systematic approach that integrates a human-centred process, existing theory and evidence, and high methodological standards. Overall, we acknowledge that we focused only on single interactive technologies that have been validated for working teams as users, and that we reviewed work only from two research streams (WOP and HCI); thus, we neglected other interactive technologies that exist outside the organizational context and in other disciplines, other possible search strategies, as well as the fact that working teams use multiple interactive technologies at the same time. However, we believe that we have taken a first step towards a more complete view of team effectiveness in the digital age, bringing WOP and HCI views and studies together. Our work provides suggestions for future work that could progress the development and implementation of interactive technologies within working teams.

Acknowledgements

We would like to thank Yanchen, Marissa, Melissa, Dasha, Valery, Lilian, Artem, Katja, Natalia and Fabio for their support at different stages of this manuscript and Hannah for her visual inspirations.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/1359432X.2023.2178904

Notes

1. A collection of individuals, who “possess one or more common goals, are brought together to perform organizationally relevant tasks, exhibit interdependencies with respect to workflow, goals, and outcomes, have different roles and responsibilities, and are together embedded in an encompassing organizational system, with boundaries and linkages to the broader system context and task environment”. (Kozlowski & Ilgen, 2006, p. 79).

2. Teamwork is defined as “a set of interrelated thoughts, actions, and feelings of each team member that are needed to function as a team and that combine to facilitate coordinated, adaptive performance and task objectives resulting in value-added outcomes”. (Salas et al., 2005, p. 562).

3. We excluded all articles that treated different interactive technologies as one ‘technology’, as this did not allow us to differentiate between them and review their impact distinctively. However, when articles investigated the impact of multiple interactive technologies separately, these were included for coding.

4. As the coders (e.g. research assistants) did not have the same experience and expertise as the authors in the research topic, in case of uncertainty (e.g. unclear with whom the interactive technology was validated), one of the authors screened the respective article ensuring that the inclusion criteria were met.

5. The results of the systematic review are based on 37 accepted papers, 38 studies, and 35 interactive technologies (see Table 1 and Appendix B). Data were extracted from each of the studies according to the criteria outlined above. However, studies did not always include information for all coding categories (e.g. team size, team membership). Therefore, tabulation of studies in each category may not always sum to the total number of articles included in the review. Furthermore, two studies did not provide information about the impact of the interactive technology on some of the constructs we coded (Gumienny et al., 2014; Heidrich et al., 2015).