Leadership in virtual work settings: what we know, what we do not know, and what we need to do

ABSTRACT

With the proliferation of flexible and remote work arrangements, researchers and practitioners are increasingly interested in leadership in virtual work settings. However, diverse conceptual understandings and methodological approaches along with partly diverging empirical findings impede simple recommendations. This review provides a systematic overview of 66 empirical studies on leadership in virtual work settings. First, we describe the various conceptual and methodological approaches found in the obtained studies. Based on this review, we categorize the studies into two main clusters, either focussing on leadership effects within highly virtual contexts only (cluster 1) or examining whether leadership effects change as a function of virtuality (cluster 2). Results revealed predominantly positive correlations between task-oriented, relational-oriented, and change-oriented leadership and follower reactions within highly virtual work contexts. Moreover, higher levels of virtuality even increased the effects of task-oriented and some relational-oriented leadership styles, while mixed findings were found for change-oriented leadership. Based on our review, we develop a research framework that includes a new integrative conceptualization of leadership in virtual work settings, as well as suggestions for standardization and study designs that might further consolidate this evolving field.

KEYWORDS:

• Leadership
• e-leadership
• virtual work
• electronically mediated collaboration
• literature review

Imagine Natalie, a committed and ambitious manager who oversees the HR department of a medium-sized insurance company. With offices on the same floor, Natalie usually sees the HR associates who are reporting to her every day. Moreover, daily team meetings help to catch up on current tasks and topics. Then, during the COVID-19 pandemic, her company decided to switch to remote work to reduce infection risks by physical distancing. Overnight, Natalie – as many other supervisors and managers worldwide (e.g., Rudolph et al., 2021) – was forced to lead her employees from a distance using predominantly electronic information and communication technologies. But what is the current state of research on leadership in such virtual work settings? Is Natalie supposed to adapt her behaviours to remain an effective leader?

Virtual work settings are characterized by high use of information and communication technologies (ICT) during collaboration, often from spatially distributed locations (Raghuram et al., 2019). In recent years, leadership in virtual work settings has received considerable attention from scholars and practitioners (e.g., Avolio et al., 2014; Brown et al., 2021; Kahai, 2013; Kahai et al., 2017), but an overview addressing the question of whether leaders should adapt their behaviours to stay successful in virtual work contexts is still lacking. This may partly be due to the heterogeneity in conceptual understanding and methodological approaches in this domain (e.g., Gibson & Gibbs, 2006; Schweitzer & Duxbury, 2010), which is difficult to integrate in meta-analyses and may be the reason why prior quantitative reviews (Brown et al., 2021) refrained from examining virtuality as a moderator of leadership effects. Therefore, we use a narrative review to examine whether leadership effects differ as a function of virtuality at work.

The current review provides a systematic overview of quantitative research on leadership in virtual work settings for epistemic and applied purposes. Based on 66 empirical studies on the relationship between leadership styles (e.g., transformational leadership and transactional leadership) and follower reactions (e.g., performance and job satisfaction) in virtual work settings, we categorize the conceptual and methodological approaches of extant studies, and critically reflect on the robustness of this research. Moreover, we derive practical suggestions for leadership in virtual work settings based on the findings obtained, also in comparison with non-virtual settings. Finally, we offer a new integrated conceptualization of leadership in virtual work settings, along with suggestions for future research in this evolving field.

This review contributes to the leadership literature in several ways. First, we provide an overview of extant studies on leadership in virtual work contexts, with a focus on studies using multi-source and multi-measurement designs. Specifically, we reveal conceptual and methodological differences across studies and categorize the reviewed studies into two clusters with different research perspectives. The studies in the first cluster considered leadership in highly virtual work contexts only, examining whether certain leadership styles “work” in virtual settings. Those studies examined different leadership constructs that we classified into task-oriented (e.g., directive leadership), relational-oriented (e.g., LMX), change-oriented (transformational leadership), and passive leadership (laissez-faire leadership). By reviewing those studies, we provide a broader and more fine-grained review as compared to prior work that focused on the more general constructs of relationship- and task-oriented leadership only (Brown et al., 2021). The studies in the second cluster compared leadership effects at different levels of virtuality, which relates to whether leadership should be adapted to the level of virtuality. Reviewing those studies extends prior reviews that did not consider virtuality as a moderator of leadership effects (Brown et al., 2021).

Third, we seek to clarify ambiguities associated with the operationalization of leadership in virtual work contexts by providing a focused but yet inclusive conceptualization. Specifically, in line with the definition of e-leadership (Avolio et al., 2014), we suggest to focus on electronically mediated leader–follower interaction as the key element, which can be measured as the number of mediated communication acts in relation to the overall communication acts. Other characteristics that have been suggested as subdimensions of virtual work in earlier research (e.g., spatial or temporal dispersion, informational capacity of involved technologies; e.g., Kirkman & Mathieu, 2005) are considered as qualifying factors of electronically mediated communication. This approach enables a more precise measurement of virtuality as a leadership context, and also a better theoretical connection with psychological process theories, such as cues-filtered-out approaches (e.g., Culnan & Markus, 1987; Daft & Lengel, 1984, 1986) or deindividuation theories (Lea & Spears, 1992; Spears & Lea, 1994; see Krämer & Winter, 2017, for a review). Relatedly, this operationalization of leadership in virtual work contexts contributes to the emerging virtuality-as-a-paradox perspective (Purvanova & Kenda, 2018, 2022) because it helps to better understand when positive or negative effects of virtuality prevail, and when opposing forces cancel each other out.

Finally, we contribute to the extant literature on leadership in virtual work contexts by providing specific suggestions for further development of this evolving field, including higher standardization of concepts and measures, advanced research designs that reflect different levels and features of virtuality, and the explicit call to consider current technological advancements.

Virtual work settings

The specific physical and social environment in which leadership occurs affects the occurrence and effectiveness of behaviours and interactions (Liden & Antonakis, 2009). Therefore, leadership scholars emphasized that the context, in which leadership occurs, needs to be considered to explain and predict leadership effects (e.g., Antonakis et al., 2003). Leadership is context-dependent, and contextual variables (e.g., organizational structures, environmental factors, demographic variables) should be considered as moderators of the relationship between leadership and follower reaction (e.g., Lowe & Gardner, 2000; Rousseau & Fried, 2001).

The rise of digital technologies has created a new socio-technological context in which large parts of today’s leadership operate (Kahai et al., 2017). Digital technologies in organizations transform how leaders and followers communicate and interact, how information is generated and shared, and how work is performed and organized around the globe (e.g., Larson & DeChurch, 2020; Rudolph et al., 2021). To describe this new work context, prior studies have used a variety of terms and conceptualizations, such as virtual work, virtual teamwork, virtual collaboration, telework, telecommuting, Internet-based work, web-based work, remote work, distributed work, flexible work, and distance work (Allen et al., 2015; Hertel et al., 2017; Raghuram et al., 2019). Table 1 provides examples of different terms and their definitions used in extant research.

Table 1. Virtual work settings: Terminologies and definitions used in the literature.

Term	Exemplar citation(s)	Definition used
Virtual team	Berry (2014)	A virtual team is a group of individuals not located in the same physical location but working towards a common goal and collaborating with other team members primarily through electronic media (Malhotra et al., 2007)
	Furumo et al. (2012)	Virtual teams can be defined as geographically or organizationally dispersed groups of individuals that communicate via information communications technology in synchronous or asynchronous modes (Powell et al., 2004)
	Gallenkamp et al. (2011)	Groups of geographically and/or organizationally dispersed co-workers that are assembled using a combination of telecommunications and information technologies to accomplish an organizational task (Townsend et al., 1998)
	Hoch and Kozlowski (2014)	Virtual teams work together over time and distance via electronic media to combine effort and achieve common goals (Bell & Kozlowski, 2002)
	Hoch and Wegge (2008)	Virtual team members are geographically dispersed and coordinate their work predominantly with electronic information and communication technologies (e.g., email, video-conferencing; e.g., Hertel et al., 2005)
(Globally) distributed team	Gajendran and Joshi (2012)	Distributed teams are characterized by geographic and time zone dispersion, reliance on electronic communication, and membership heterogeneity (Bell & Kozlowski, 2002; Gibson & Gibbs, 2006; Martins et al., 2004)
Electronic groups, computer-supported groups	Kahai et al. (2004)	Such groups may employ a Group Decision Support System ... and include virtual groups, whose members are located remotely, as well as groups whose members are located in the same place (Nunamaker et al., 1997; Townsend et al., 1998).
	Sosik et al. (1998)	Group members interact electronically through communication of text-, graphics., video., or voice-based information or through a combination of electronic and ftf interaction
Telework, telecommuting	Konradt et al. (2003)	Telework, as a decentralized computer-mediated work form that enables employees to work in locations distant from the main organization (e.g., Di Martino & Wirth, 1990)
	Madlock (2012)	Working at one’s home or another location where employees use computers and communication technology to communicate with the main office, supervisors, co-workers, and customers (Gibson et al., 2002)
	Niedle (2012)	Telecommuting, also referred to as remote work or telework, is a work arrangement that allows employees to perform tasks from home or remote offices that are normally done in a central workplace using electronic media to communicate with others inside and outside the organization (Bailey & Kurland, 2002; Baruch, 2001; Feldman & Gainey, 1997)
Remote work, distributed work, virtual work	Golden and Veiga (2005)	Involving performance of work assignments away from the traditional office, either from home or somewhere else on the road
	Wolter (2017)	Individuals who works remotely as an individual who conducts work outside of a centrally located work environment

We use the term virtual work throughout our review, which has been defined as “geographic dispersion and dependence on technology in work-related interactions between employees” (Raghuram et al., 2019). Early studies often contrast virtual work with face-to-face work, whereas more recent work considers virtuality as a continuum of varying degrees and with different subdimensions (e.g., Bell & Kozlowski, 2002; Gibson & Gibbs, 2006; Hertel et al., 2017; Schweitzer & Duxbury, 2010). Examples of such subdimensions include electronic mediation of interaction, spatial dispersion, temporal dispersion or interaction asynchronicity, and informational capacity of technologies (e.g., Bell & Kozlowski, 2002; Hertel et al., 2005; Kirkman & Mathieu, 2005). Moreover, accessibility, interactivity, and reprocessabilty of information together with automation of interaction have been suggested as functional aspects of virtual work contexts that can vary independently (Hertel et al., 2017). However, most extant empirical work so far has focused on electronic mediation of interaction and on spatial dispersion as subdimensions of virtual work (Raghuram et al., 2019).

Notably, these subdimensions are related to somewhat different psychological processes, and therefore can have different implications for workers’ experience and behaviour at work. For instance, spatial distance between leader and followers can decrease perceived control or support, whereas temporal distance (asynchronous communication) may provide additional latitude for workers’ time management. To date, such a more detailed linkage between features of virtual work and psychological processes is still at its infancy (e.g., Hertel et al., 2017; Krämer & Winter, 2017). The currently most prominent theories focus on electronic mediation of information as a core aspect of virtual work. Theories following a “cues-filtered-out” perspective (e.g., Culnan & Markus, 1987; Daft & Lengel, 1984, 1986) note that virtual work often comes with a reduction of social cues that might lead to misunderstandings and reduced socio-emotional contact between actors. While such cue reduction can vary considerably across technologies (e.g., email vs. video conferencing), the general perspective is that virtual interaction is hampered by fewer transmissions of social information as compared to a “gold” standard of non-virtual face-to-face interaction (e.g., Krämer & Winter, 2017).

Theories following a “deindividuation” perspective (e.g., Lea & Spears, 1992; Spears & Lea, 1994) also refer to the electronic transmission of information as a core feature, but with somewhat different implications. Specifically, this perspective suggests that a reduced availability of individuating cues in virtual contexts makes followers more responsive to cues that emphasize larger social entities, such as a team or the whole organization (e.g., Joshi et al., 2009; Kahai et al., 2013). This can have considerable implications for leadership addressing team processes and organizational identification and goals. Below, we will refer to these different perspectives when summarizing and integrating the various findings of our review.

Leadership in virtual work settings

In the light of the described different research traditions and conceptualizations of virtual work, it is not surprising that leadership in virtual settings is also discussed under different labels, such as e-leadership, virtual leadership, virtual team leadership, or remote management, among other terms (see Table 2). However, similar to virtual work settings, most definitions of leadership focus either on electronically mediated communication, spatial distance, or both. For instance, Avolio et al. (2014) highlight that electronically mediated social influence takes place when leader and follower are spatially separated, but also when they work in the same location but interact via information and communication technologies.

Table 2. Leadership in virtual work settings: Terminologies and definitions used in the literature.

Term	Exemplar citation(s)	Definition used
E-leadership	Hambley et al. (2007)	A social influence process mediated by advanced information technologies to produce changes in attitudes, feelings, thinking, behaviour, and/or performance of individuals, groups, and/or organizations (Avolio et al., 2001)
Virtual leadership^a	Berry (2014)	The virtual leader is a leader who is responsible for the management of employees or work groups who are dispersed geographically and rely primarily upon electronic media for communication and collaboration (Malhotra et al., 2007)
Virtual team leadership^b	Mawanda (2012)	The ability to successfully organize and facilitate group members located in varying geographical locations and time zones
Remote leadership, electronically-mediated leadership^c	Kelloway et al. (2003)	Leadership interactions that are characterized by electronically-mediated communication between geographically and physically isolated leaders and followers
Remote management	Watson (2007)	The term “remote management” refers to managing at a distance where employees spend at least some portion of their time working in a different location than their manager, which can include customer/partner locations, home, other company buildings and/or different areas within the same company building

^aSimilar terms used, but not defined: virtual work leadership, leadership of telecommuters, leadership in virtual environments, leadership of virtual workers, leadership in a GDSS context, leadership in electronically enabled work contexts.

^bSimilar terms used, but not defined: virtual project leadership, electronic group leadership, leadership in (globally) distributed or (spatially) dispersed teams or groups, leadership of computer-supported groups, leadership in virtual world teams, global team leadership.

^cSimilar terms used, but not defined: distant leadership.

In addition to the described variety of conceptualizations, extant studies on leadership in virtual settings are quite diverse in their study design and sampling. As a consequence, the results are difficult to be compared, aggregated, and theoretically integrated (e.g., Allen et al., 2015; Larsen & Bong, 2016). Initial reviews and meta-analyses have focused on specific domains, such as virtual teams (Brown et al., 2021; Gilson et al., 2015; Hertel et al., 2005; Purvanova & Kenda, 2022) or telework (Allen et al., 2015; Bailey & Kurland, 2002; Gajendran & Harrison, 2007). Raghuram et al. (2019) provide an integration of research across different domains of virtual work; however, their review does not explicitly address leadership in virtual settings. Reviews that specifically address leadership in virtual settings are conceptual papers on e-leadership (Avolio et al., 2001, 2014; Kahai, 2013; Kahai et al., 2017). We agree with those authors’ approach to define leadership in virtual settings as “a social influence process embedded in both proximal and distal contexts mediated by advanced information technology that can produce a change in attitudes, feelings, thinking, behaviour, and performance” (Avolio et al., 2014, p. 107).

In the current review, we focus on the effectiveness of specific leadership styles in virtual work contexts. In doing so, we categorize the various leadership styles in the reviewed studies following established taxonomies (e.g., DeRue et al., 2011; Yukl et al., 2002) into four broader types: Task-oriented leadership, relational-oriented leadership, change-oriented leadership, and passive leadership. Additional suggested types of leadership, such as boundary spanning leadership or destructive leadership (e.g., Lyubykh et al., 2022; Yuki, 2012), are certainly also relevant for virtual work contexts; however, we found no empirical studies that have already addressed these types of leadership explicitly in virtual contexts.

Task-oriented leadership focuses on the coordination and monitoring of work processes, including initiating structures such as defining task and role assignments, goal setting and feedback processes, as well as performance monitoring and contingent reward (e.g., DeRue et al., 2011). We considered directive leadership, management by objectives, and transactive leadership as predominantly task-related types of leadership. Relational-oriented leadership focuses on the management of positive work relationships with followers, showing consideration and respect, and building trust. Moreover, relational-oriented leaders follow rather participative leadership approaches and are open to input from their followers (e.g., DeRue et al., 2011; Yukl et al., 2002). Relational-oriented leadership has been noted to be particularly important in work arrangements with reduced contact with and/or among followers (e.g., Hertel et al., 2005; Hoch & Kozlowski, 2014). We considered LMX, participative leadership and shared leadership as well as authentic leadership and servant leadership as predominantly relational-oriented types of leadership. Change-oriented leadership includes leadership that initiates and supports transition and change in organizations, including challenging the status quo, developing visions for change, and encouraging innovation (e.g., DeRue et al., 2011). Given that increasing amounts of virtuality at work are often a major and/or ongoing change in itself, change-related leadership might be particularly relevant in virtual work contexts. Among the leadership styles addressed in the studies covered by this review, we consider transformational leadership as a predominantly change-oriented type of leadership even though some aspects of transformational leadership (individualized consideration) are relational-oriented. However, most other facets of transformational leadership address the development and communication of a clear vision as well as facilitating innovation (DeRue et al., 2011). Finally, passive leadership as a fourth type of leadership refers to leader inaction. Instead of being suggested as a true alternative, this category is usually included as some sort of baseline to contrast with other leadership styles (e.g., Bass, 1990). However, the success of self-organizing online groups and communities (e.g., Open Source Software developers; Crowston et al., 2008; Hertel et al., 2003) might suggest that virtual work contexts require less personal leadership, therefore we also consider available empirical findings on passive leadership in this review.

A recent meta-analysis (Brown et al., 2021) already examined the associations of task-oriented and relational-oriented leadership with performance in virtual team contexts. The results revealed positive correlations for both types of leadership. Moreover, these relationships were moderated by task interdependence, team size, team type, and the methodological approach to measure team performance (i.e., self-reports vs. non-self-reports). However, apart from concentrating on team settings and two leadership types, this prior meta-analysis focused only on leadership effects in highly virtual work settings, and did not compare these effects with leadership in low or non-virtual work settings. Thus, it remains open to what extent leadership might have to be adapted to the relative degree of virtuality.

The present review

The present review summarizes concepts, methods, and results of extant research on leadership in virtual work settings. To identify relevant studies, we used two strategies. First, we conducted an extensive electronic literature search in several databases (PsycInfo, Academic Search Premier, Business Source Premier, Web of Science, and Google Scholar) using keywords such as “leader”, “e-leader”, and “virtual”. Second, we apply back-tracing by scanning the reference lists of key reviews on e-leadership (Avolio et al., 2014; Kahai et al., 2017) and related domains such as virtual work/teams (Gilson et al., 2015; Hertel et al., 2005; Raghuram et al., 2019) and telework (Bailey & Kurland, 2002) for relevant sources. These two search strategies resulted in 4.176 studies potentially eligible for the review. For inclusion in this systematic review, studies had to meet five criteria: (1) examination of leadership in a virtual work setting (notably, we included student samples simulating organizational context and studies conducted in a collaborative gaming environment); (2) examination of established leadership styles rather than investigating leader traits, single behaviours, or skills; (3) provision of quantitative data on the relations of interest; (4) written in English language; (5) full-text accessibility. In case of unpublished studies that were not publicly available, we sent two requests to the author(s), or, in the case of dissertations, to the supervisor if contact data were available. The same procedure was used when relevant statistical information was missing. Articles or requested data that were not sent as a full text were not considered in this review. A total of 66 studies met the criteria for inclusion in our review. If multiple studies were based on the same sample and variables, we only included the most comprehensive data.¹

What we know, what we don’t know

Table 3 provides a detailed overview on the features of the considered studies, including information on study design, samples, leadership style, follower reactions, and the specifics of the virtual work setting. As can be seen, studies varied considerably in the operationalizations of virtuality and general research design, among other things. As a consequence, a quantitative aggregation of the results, e.g., in a meta-analysis, is difficult and might even lead to misinterpretations comparing “apples with oranges”. Therefore, we summarize the findings descriptively, paying particular attention to the methodological rigour of the applied methodologies.

Table 3. An overview of studies on leadership in virtual work settings.

Source	Design & sample	Leadership				Outcome		Virtuality
		Leadership style(s)	Source(s)	Measurement level	Locus	Variable(s)	Source	Context	Operationalization/measurement	Dimensions/indicators^a
Andressen et al. (2012)	Field study, cross sectional professionals that collaborate under different degrees of virtuality	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Self-leadership, job performance, commitment, motivation	Follower & supervisor ratings	Virtual/hybrid teams	Team leader and team virtuality, continuous measurement, dichotomous moderation analysis using median split into high & low level virtuality	Frequency of computer-mediated communication, physical distance, combination of both indicators into virtuality index
Arnold (2008)	Field study, cross-sectional, professionals working in virtual projects	Transformational, transactional, passive leadership	Supervisor ratings	Individual level	Hierarchical leadership	Project success	Supervisor ratings	Virtual project	Team virtuality, no measurement, virtual project as study context
Aslam (2010)	Field study, cross-sectional, professionals working in virtual projects	Task-, relationship-oriented leadership	Follower ratings	Individual level	Hierarchical leadership	Empowerment, service orientation	Follower ratings	Virtual project	Team virtuality, continuous measurement of virtuality, dichotomous division of sample into high & low level virtuality, virtual project as study context	Measure of virtuality includes frequency of computer-mediated communication, geographic dispersion
Balthazard et al. (2007)	Experiment, cross-sectional student ad hoc groups	Transformational leadership	Follower ratings	Team level	Hierarchical leadership	Team error, synergy, cohesion, solution acceptance	Follower & supervisor ratings, objective data	Virtual teams	Team virtuality, manipulation of virtuality (virtual vs. face-to-face as control group)	General virtuality
Balwant (2017)	Field study, cross-sectional, online student project teams	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Work engagement	Follower ratings	Virtual/hybrid project	Leader virtuality, continuous measurement	Physical distance, social distance, and frequency of computer-mediated communication, combination of indicators into virtuality index
Berry (2014)	Field study, cross-sectional, professionals, face-to-face interaction with supervisors and co-workers one time per month or less	Servant leadership	Follower ratings	Individual level	Hierarchical leadership	Commitment, job satisfaction	Follower ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context
Bogler et al. (2013)	Field study, cross-sectional, online students without face-to-face interaction	Transformational, transactional, passive leadership	Follower ratings	Individual level	Hierarchical leadership	Satisfaction, participation, performance	Follower ratings	Virtual work (distance education supported by online discussion forum)	General virtuality, no measurement, virtual work as study context
Chang and Lee (2013)	Field experiment, cross-sectional, online students without face-to-face interaction	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Learning performance	Objective data	Virtual team (distance education supported by synchronous learning platform	General virtuality, no measurement, virtual work as study context
Duffy (2018)	Field study, cross-sectional, professionals without supervisor at their location	Transformational, transactional, passive leadership	Follower ratings	Individual level	Hierarchical leadership	Satisfaction, leader effectiveness, extra effort	Follower ratings	Telework/virtual work	Leader virtuality, no measurement, virtual work as study context
Eveleth and Eveleth (2010)	Field study, cross-sectional, online game players without face-to-face interaction	LMX	Follower ratings	Individual level	Hierarchical leadership	Identification with team, satisfaction with team, team performance	Follower ratings	Virtual work (online game supported by virtual world)	General virtuality, no measurement, virtual work as study context
Fuller and Harding (2015)	Experiment, cross-sectional, online students without/limited face-to-face interaction	Shared leadership	Follower ratings	Individual level	Shared leadership	Job performance	Follower ratings & objective data	Virtual work (distance education supported by online discussion tool)	General virtuality, no measurement, virtual work as study context
Furumo et al. (2012)	Quasi experiment, multiple measure points, online students without face-to-face interaction	Directive, supportive leadership	Manipulated	Individual level	Hierarchical leadership	Participation, trust	Follower ratings & objective data	Virtual work (distance education supported by online discussion tool)	General virtuality, no measurement, virtual work as study context
Gajendran and Joshi (2012)	Field study, multiple measure points, professionals that collaborate under different degrees of virtuality	LMX	Follower ratings	Individual level	Hierarchical leadership	Member influence on team decisions, team innovation	Follower & supervisor ratings	Virtual/hybrid teams	Team virtuality, continuous measurement for moderation analysis	Team dispersion (geographic and time zone dispersion)
Gallenkamp et al. (2011)	Field study, multiple measure points, online game players without face-to-face interaction	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Trust in leader, cohesion, team performance	Follower ratings & objective data	Virtual work (online game supported by virtual world)	General virtuality, no measurement, virtual work as study context
Goh and Wasko (2012)	Field study, multiple measure points, online game players without face-to-face interaction	LMX	Follower & supervisor ratings	Individual level	Hierarchical leadership	Identification, empowerment, obligation, norms, performance, trust	Follower, supervisor& peer ratings & objective data	Virtual work (online game supported by virtual world)	General virtuality, no measurement, virtual work as study context
Golden and Veiga (2008)	Field study, multiple measure points, professionals that spend an average of 25% of their work in the virtual mode	LMX	Follower ratings	Individual level	Hierarchical leadership	Commitment, job satisfaction, performance	Follower ratings	Telework/virtual work	General virtuality, continuous measurement for moderation analysis	General virtuality
Hahm (2017)	Field study, cross-sectional, professionals working in virtual projects	Authentic leadership	Follower ratings	Individual level	Hierarchical leadership	Information sharing, empowerment, creativity	Follower ratings	Virtual/hybrid project	General virtuality, no measurement, virtual work as study context
Hambley et al. (2007)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Team cohesion, constructive interaction, team performance	Follower ratings & objective data	Virtual teams	Team virtuality, manipulation of virtuality (face-to-face vs. videoconference vs. chat) for analysis of interaction effects	General virtuality, media richness
Hertel et al. (2004)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Management by objectives	Follower ratings	Team level	Hierarchical leadership	Valence, instrumentality, self-efficacy, trust, team effectiveness	Follower & supervisor ratings	Virtual/hybrid teams	Team virtuality, measurement for sample description, virtual work as study context
Hill and Bartol (2016)	Field study, multiple measure points, professionals in virtual teams that collaborate under different degrees of virtuality	Empowering leadership	Follower ratings	Team level	Hierarchical leadership	Collaboration, team performance	Follower ratings	Virtual/hybrid teams	Team virtuality, continuous measurement for moderation analysis	Team geographic dispersion
Hill et al. (2014)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	LMX	Follower ratings	Individual level	Hierarchical leadership	Empowerment, commitment, job satisfaction, performance	Follower ratings	Telework/virtual work	Team virtuality, continuous measurement for moderation analysis	Frequency of computer-mediated communication
Hite (2009)	Field study, multiple measure points, online students with limited face-to-face interaction	Task-, relationship-oriented leadership	Follower ratings	Individual level	Hierarchical leadership	Leader effectiveness, identification, counterproductive behaviour	Follower ratings	Virtual work (distance education supported by online discussion forum)	General virtuality, no measurement, virtual work as study context
Hoch and Kozlowski (2014)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Transformational, shared leadership, LMX	Follower ratings	Team level	Hierarchical & shared leadership	Team performance	Supervisor ratings	Virtual/hybrid teams	Team virtuality, continuous measurement for moderation analysis	Geographic dispersion, frequency of computer-mediated communication, cultural background, combination of indicators into virtuality index
Hoch and Wegge (2008)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Shared leadership	Follower ratings	Team level	Shared leadership	Team performance	Supervisor ratings	Virtual/hybrid teams	Team virtuality, measurement for sample description, virtual work as study context
Howell and Hall-Merenda (1999)	Field study, multiple measure points, professionals that collaborate under different degrees of virtuality	Transformational, transactional, passive leadership, LMX	Follower ratings	Individual level	Hierarchical leadership	Performance, LMX	Supervisor ratings	Telework/virtual work	Leader virtuality, continuous measurement, dichotomous division of sample into high & low leader virtuality for moderation analysis	Physical distance
Howell et al. (2005)	Field study, multiple measure points, professionals in virtual teams that collaborate under different degrees of virtuality	Transformational, transactional leadership	Follower ratings	Team level	Hierarchical leadership	Performance	Objective data	Virtual/hybrid teams	Leader virtuality, continuous measurement, dichotomous division of sample into high & low leader virtuality for moderation analysis	Physical distance
Hoyt and Blascovich (2003)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Satisfaction with leader, team cohesion, quantitative & qualitative performance	Follower ratings & objective data	Virtual teams	General virtuality, manipulation of virtuality (face-to-face vs. telephone vs. virtual world) for analysis of interaction effects	General virtuality, media richness
Huang et al. (2009)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Supportive & critical communication	Follower ratings & objective data	Virtual teams	General virtuality, manipulation of virtuality (chat vs. virtual world) for analysis of interaction effects	General virtuality, media richness
Huang et al. (2010)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Cooperative climate, task cohesion	Follower ratings & objective data	Virtual teams	General virtuality, manipulation of virtuality (chat vs. virtual world) for analysis of interaction effects	General virtuality, media richness
Joshi et al. (2009)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Inspirational leadership	Follower ratings	Individual level	Hierarchical leadership	Commitment to team, trust in team member	Follower & supervisor ratings	Virtual/hybrid teams	Team virtuality, continuous measurement for moderation analysis	Team dispersion (temporal, configural, spatial, cross-national dispersion & proportion of telecommuters, combination of indicators into virtuality index)
Jury (2008)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Transformational, transactional leadership	Follower ratings	Individual level	Hierarchical leadership	Role clarity, job satisfaction, performance, trust in leader	Follower & supervisor ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context
Jury et al. (2005)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Role clarity, knowledge sharing, performance, trust in leader	Follower & supervisor ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context
Kahai & Avolio (2007)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Participation, balance of arguments, post discussion deviation, change in intention	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Kahai et al. (1997)	Experiment, cross-sectional, student ad hoc groups	Participative, directive leadership	Manipulated	Team level	Hierarchical leadership	Supportive & critical communication, attitude towards leader, creativity	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS)	General virtuality, no manipulation, virtual work as study context
Kahai et al. (2003)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Participation, cooperation, creativity, team efficacy, task satisfaction, satisfaction with leader	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Kahai et al. (2004)	Experiment, cross-sectional, student ad hoc groups	Participative, directive leadership	Manipulated	Team level	Hierarchical leadership	Job satisfaction, satisfaction with leader, participation, performance	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS)	General virtuality, no manipulation, virtual work as study context
Kahai et al. (2012)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Feedback positivity	Follower ratings & objective data	Virtual teams	General virtuality, manipulation of virtuality (chat vs. virtual world) for analysis of interaction effects	General virtuality, media richness
Kahai et al. (2013)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Cognitive effort, change in decision quality, discussion satisfaction	Follower ratings	Virtual teams (supported by virtual world)	General virtuality, no manipulation, virtual work as study context
Kahai et al. (2013)	Field study, cross-sectional, professionals in virtual project teams that collaborate under different degrees of virtuality	Transformational leadership	Follower & supervisor ratings	Team level	Hierarchical leadership	Collaboration quality, team performance	Follower & supervisor ratings	Virtual/hybrid teams	Team virtuality, continuous measurement for moderation analysis	Media richness (extent of lean media use)
Kelloway et al. (2003)	Experiment, cross-sectional, students without face-to-face interaction with the supervisor	Transformational leadership	Manipulated	Individual level	Hierarchical leadership	Extra effort, performance	Follower ratings & objective data	Virtual work (supported by email)	Leader virtuality, no manipulation, virtual work as study context
Konradt et al. (2003)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Management by objectives	Follower ratings	Individual level	Hierarchical leadership	Stress, job satisfaction	Follower ratings	Telework/virtual work	General virtuality, measurement for sample description and control group, virtual work as study context
Lyon (2003)	Experiment, cross-sectional, students without face-to-face interaction with the supervisor	Transformational, transactional leadership	Manipulated	Individual level	Hierarchical leadership	Job satisfaction, well-being, performance quality & quantity	Follower ratings & objective data	Virtual work (supported by email)	Leader virtuality, no manipulation, virtual work as study context
Madlock (2012)	Field study, cross-sectional, professionals without/limited face-to-face interaction with the supervisor	Task-, relationship-oriented leadership	Follower ratings	Individual level	Hierarchical leadership	Job satisfaction, communication satisfaction, commitment	Follower ratings	Telework/virtual work	Leader virtuality, measurement for sample description, virtual work as study context
Mawanda (2012)	Field study, cross-sectional, professionals without/limited face-to-face interaction	Transformational, transactional leadership	Follower ratings	Individual level	Hierarchical leadership	Job satisfaction, extra effort, leader effectiveness	Follower ratings	Telework/virtual work	Team virtuality, no measurement, virtual work as study context
McCorkle (2012)	Field study, cross-sectional, professionals in virtual project teams that collaborate under different degrees of virtuality	Transformational leadership	Supervisor ratings	Individual level	Hierarchical leadership	Project performance	Supervisor ratings	Virtual project	Project virtuality, dichotomous measurement to divide sample into virtual & face-to-face project teams for moderation analysis	General virtuality
Moore (2008)	Field study, cross-sectional, professionals without/limited face-to-face interaction with the supervisor	Transformational leadership, LMX	Follower ratings	Individual level	Hierarchical leadership	Motivation	Follower ratings	Telework/virtual work	Leader virtuality, measurement for sample description, virtual work as study context
Muethel et al. (2012)	Field study, cross-sectional, professionals in virtual project teams that collaborate under different degrees of virtuality	Shared leadership	Follower ratings	Team level	Shared leadership	Team effectiveness	Supervisor ratings	Virtual/hybrid teams	Team virtuality, measurement for sample description, virtual work as study context
Niedle (2012)	Field study, cross-sectional, professionals (supervisors of co-located and remote followers)	LMX	Supervisor ratings	Individual level	Hierarchical leadership	In-role behaviour, OCBs	Supervisor ratings	Telework/virtual work	Leader virtuality, continuous measurement for moderation analysis	Computer-mediated communication (frequency, media richness, combination into virtuality index)
Pearce et al. (2004)	Field study, multiple measure points, professionals in an online educational program without face-to-face interaction	Transformational, transactional, empowering, directive, shared leadership	Follower ratings	Individual level	Hierarchical & shared leadership	Potency, social integration, problem solving quality, effectiveness	Follower ratings	Virtual teams	General virtuality, no measurement, virtual work as study context
Pham (2016)	Field study, cross-sectional, professionals without/limited face-to-face interaction with the supervisor	Task-, relationship-oriented leadership	Follower ratings	Individual level	Hierarchical leadership	Work engagement	Follower ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context
Purvanova and Bono (2009)	Experiment, cross-sectional, student ad hoc groups	Transformational leadership	Manipulated, follower ratings	Individual & team level	Hierarchical leadership	Team performance, project satisfaction	Follower ratings & objective data	Virtual teams	Team virtuality, manipulation of virtuality (virtual vs. face-to-face as control group) for moderation analysis	General virtuality
Robert and You (2018)	Field study, multiple measure points, online student teams without face-to-face interaction	Shared leadership	Follower ratings	Individual level	Shared leadership	Satisfaction with team, trust in team, team performance, autonomy	Follower ratings	Virtual teams	Team virtuality, no measurement, virtual work as study context
Ruggieri (2009)	Experiment, multiple measure points, student ad hoc teams	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Job satisfaction	Follower ratings	Virtual teams	General virtuality, no measurement, virtual work as study context
Ryan (2008)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Transformational, transactional, passive, shared leadership	Follower ratings	Individual level	Hierarchical & shared leadership	OCBs, team performance, perceived organizational support	Follower ratings	Virtual/hybrid teams	General virtuality, no measurement, virtual work as study context
Salvaggio (2016)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Transformational leadership, LMX	Follower ratings	Individual level	Hierarchical leadership	LMX, job satisfaction, commitment	Follower ratings	Telework, virtual work	Leader virtuality, continuous measurement for moderation analysis	Computer-mediated communication (frequency, media richness as separate indicators)
Sha and Chang (2012)	Field study, cross-sectional, professionals (part-time students) with virtual work experience	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Interpersonal helping, commitment to team, knowledge sharing, relational capital	Follower ratings	Virtual/hybrid teams	Team virtuality, no measurement, virtual work as study context
Sinani (2016)	Field study, cross-sectional, professionals without/limited face-to-face interaction	Participative leadership	Follower ratings	Individual level	Hierarchical leadership	Job satisfaction	Follower ratings	Virtual/hybrid teams	General virtuality, no measurement, virtual work as study context
Sosik (1997)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Extra effort, performance, satisfaction with leader, creativity, supportive & critical communication	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS with face-to-face introduction)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Sosik et al. (1997)	Experiment, multiple measure points, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Team potency, effectiveness	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Sosik et al. (1998)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Team level	Hierarchical leadership	Creativity	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS with face-to-face introduction)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Sosik et al. (1999)	Experiment, cross-sectional, student ad hoc groups	Transformational, transactional leadership	Manipulated	Individual level	Hierarchical leadership	Flow, creativity	Follower ratings & objective data	Virtual teams (co-located teams interacting via GDSS with face-to-face introduction)	Team virtuality, manipulation of anonymity for analysis of interaction effects	Anonymity
Watson (2007)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Task-, relationship-oriented leadership	Follower ratings	Individual level	Hierarchical leadership	Satisfaction with leader, commitment	Follower ratings	Telework/virtual work	Leader virtuality, continuous measurement, dichotomous division of sample into high & low leader virtuality for moderation analysis	Physical distance
Whitford and Moss (2009)	Field study, cross-sectional, professionals that collaborate under different degrees of virtuality	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	Work engagement, job satisfaction	Follower ratings	Telework/virtual work	Leader virtuality, dichotomous measurement, division of sample into virtual and non-virtual follower	Physical distance (supervisor and follower work in same/different location)
Wolter (2017)	Field study, cross-sectional, professionals without face-to-face interaction	Transformational, transactional, passive leadership	Follower ratings	Individual level	Hierarchical leadership	Motivation	Follower ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context
Wong and Berntzen (2019)	Field study, multiple measure points, professionals in virtual teams that collaborate under different degrees of virtuality	Transformational leadership	Follower ratings	Individual level	Hierarchical leadership	LMX	Follower ratings	Telework/virtual work	General virtuality, continuous measurement for moderation analysis	Frequency of computer-mediated communication
Zhang et al. (2012)	Field study, cross-sectional, professionals in virtual teams that collaborate under different degrees of virtuality	Management by objectives	Follower ratings	Individual level	Hierarchical leadership	Motivation, satisfaction with leader	Follower ratings	Telework/virtual work	General virtuality, no measurement, virtual work as study context

^aNot applicable for studies that assume virtual work settings as overall study context.

Methodological approaches and sampling

The 66 studies included in this review break down into 44 correlational field studies and 22 experimental studies. Notably, these two types of studies were partly confounded with specific sampling and methodological approaches. For instance, all experimental studies relied on student participants, with known limitations in generalizability. In contrast, most field studies (n = 37) relied on active workers as participants, while only four studies relied on students and three studies on online gamers. Apart from that, the majority of both field studies (n = 35) and experimental studies (n = 19) applied cross-sectional designs, with limited opportunities to examine dynamic effects and causal relationships. Only few studies used multiple measurements. Among the field studies, for example, Howell and Hall-Merenda (1999) used two surveys with a 18-month time lag to examine the relationships of transformational, transactional, and passive leadership (time 1) with job performance (time 2). Among the experimental studies, Sosik et al. (1997) examined effects of transformational and transactional leadership on team effectiveness at two measurement points, with a one-week time lag using a computer-based group decision support system.

In addition to the restrictions due to the designs of the obtained studies, sources of information were also often limited. The majority of the obtained field studies (n = 26) relied on single source and self-report questionnaire data only, with known limitations such as common method biases and endogeneity (e.g., Antonakis et al., 2010, 2014; Podsakoff et al., 2012). This observation is not intended to devalue this important pioneering research. However, the related findings have to be interpreted with caution, and await replications and extensions in ecologically valid multi-measure studies. Nevertheless, 16 field studies did include additional data sources to assess dependent variables, such as supervisory ratings, and two of them included objective data (e.g., profit, revenue, and productivity ratio; Howell et al., 2005). Objective data were also used in all experimental studies for at least one outcome variable, usually performance measures as part of the experimental task (e.g., Hambley et al., 2007; Hoyt & Blascovich, 2003).

Apart from the described differences in sampling and research design, the obtained studies also differed in their general perspective on leadership in virtual work settings (see Table 3 for an overview). We categorize these perspectives into two major clusters, each connected with a somewhat different research question: The first cluster includes studies that examine specific leadership styles within highly virtual settings only, examining whether these leadership styles are positively related to desirable follower reactions in this specific work context. Figure 1 illustrates the approach of studies in cluster 1. This perspective, also applied in the meta-analysis by Brown et al. (2021), answers the question whether specific leadership “works” in highly virtual contexts. In contrast, the second cluster includes studies that considered virtuality as a moderating condition of leadership effects. These studies address the question of whether the effectiveness of specific leadership types is qualified by the level of virtuality, which also relates to the question of whether leadership should be adapted to the level of virtuality at work. Figure 2 shows the approach of studies in cluster 2. Next, we report the various findings separately for each of the two clusters, sorted according to the introduced taxonomy of leadership types (DeRue et al., 2011; Yukl et al., 2002) within the two clusters.

Figure 1. Leadership styles addressed in studies considering only (one level of) highly virtual work settings (cluster 1).

Figure 2. Leadership styles addressed in studies considering the degree of virtuality as a moderator or comparing virtual and non-virtual work settings (cluster 2).

Cluster 1: leadership effectiveness in virtual work settings

The first cluster included 36 studies on the correlation between specific leadership styles and follower reactions in virtual settings only, without comparing different degrees of virtuality. With respect to task-oriented leadership, 16 field studies in this cluster revealed positive relationships between task-oriented leadership styles (e.g., management by objectives and transactional leadership) and follower reactions, such as job performance and job satisfaction (e.g., Hertel et al., 2004; Konradt et al., 2003; Wolter, 2017). Similar findings were obtained with respect to relational-oriented leadership (e.g., participative leadership and LMX), with 15 field studies showing positive correlations with follower performance and job attitudes (e.g., Hill et al., 2014; Madlock, 2012). Although most of these field studies relied on self-report data and cross-sectional designs, Goh and Wasko (2012) used multiple data sources to measure times in their study on teams in computer-simulated environments, finding positive correlations between LMX (follower ratings) and followers’ performance assessed from multiple sources (follower, leader, and peer ratings) after 8 weeks. Notably, the strength of this correlation (r = .61) is considerably higher than correlations reported for LMX and performance in non-virtual settings (r between .28 and .30 according to meta-analyses; e.g., Dulebohn et al., 2012).

In general, the obtained results on task-related and relational-related leadership types correspond well to the meta-analytic findings of Brown et al. (2021), who also revealed positive overall results for task-oriented and relational-oriented leadership in the context of virtual teams. However, extending these findings, 12 studies in cluster 1 also provided evidence for positive correlations between change-oriented leadership (i.e., transformational leadership) and follower reactions, such as performance and job satisfaction (e.g., Mawanda, 2012; Wolter, 2017). Finally, four field studies addressed correlations between passive leadership (i.e., laissez-faire leadership) and follower reactions in highly virtual work contexts. The results revealed negative associations between laissez-faire leadership and follower reactions, such as job performance (Arnold, 2008; Ryan, 2008) and job satisfaction (Duffy, 2018) similar to non-virtual settings. Thus, these findings provide no indication that leadership requirements might be reduced in virtual contexts.

In addition to the described field studies, cluster 1 also includes eight experimental studies on leadership in highly virtual settings. These studies also showed positive relationships of task-oriented (i.e., transactional) leadership (e.g., Kahai et al., 2013; Lyon, 2003), relational-oriented (i.e., participative) leadership (Kahai et al., 2004), and change-oriented (i.e., transformational) leadership (e.g., Kahai et al., 2013; Lyon, 2003) with follower satisfaction. Additionally, results showed that transformational leadership increased follower performance (Chang & Lee, 2013; Kelloway et al., 2003) and satisfaction (Ruggieri, 2009) more strongly than transactional leadership. While these studies provide opportunities for causal conclusions due to the experimental variation of leadership, it should be noted that all relied on student samples and simulations of work rather than true occupational settings. Moreover, most of these studies used cross-sectional designs. As a notable exception, Furumo et al. (2012) employed multiple measurement points and multi-source data. Their experimental study compared directive and supportive leadership in virtual teams during a semester-long online student project, which might be considered as a contrast between task- and relational-oriented leadership. Results indicated that supportive leadership initially evoked higher levels of participation and trust by followers as compared to directive leadership; however, this difference disappeared over time.

Together, the studies in cluster 1 suggest that, on average, all active leadership types considered are positively correlated with desired follower reactions in virtual work settings, and thus seem to “work” in these contexts. These results challenge both a more pessimistic view that lean communication environments might limit supervisors’ potential to influence their followers (Purvanova & Bono, 2009) and the – somewhat more optimistic – assumption that virtuality might replace leadership by triggering increased self-management of followers. Rather, it seems that followers are able to recognize and interpret supervisory behaviours regardless of reduced communication cues.

From a more sceptical perspective, however, the fact that most studies in cluster 1 used cross-sectional designs and self-reported data might have led to an overestimation of correlations between leadership and follower reactions (e.g., Podsakoff et al., 2012). In addition, although all studies addressed leadership in highly virtual settings, the nature of these settings and applied technologies varied considerably, and many studies provided imprecise or even no information on the type and degree of virtuality (e.g., proportion of electronically mediated communication), making more differentiated analyses difficult. Perhaps most importantly, the studies of cluster 1 are silent on the question of if and to what extent leadership might have to be adapted to the level of virtuality. This question is addressed in the studies of cluster 2.

Cluster 2: virtuality of work as a moderator of leadership effectiveness

The second cluster included 30 studies on specific leadership styles that considered different degrees of virtuality as a dichotomous or continuous moderator, either measuring or manipulating the level of virtuality, or including a non-virtual comparison condition. These studies not only examined the correlation between leadership styles and follower reactions (i.e., main effects) but also whether these relationships are qualified by the degree of virtuality, addressing the question of whether leadership styles should be adapted to the level (or kind) of virtual work. We found three different ways to operationalize virtual work among the studies of cluster 2. First, studies modelled virtual work either as a dichotomous moderator by dividing the sample into a co-located and a dispersed sample, or by assessing virtual work as a continuous moderator variable. Second, studies measured virtual work either with a single indicator, or by considering a dimensional concept with facets such as spatial distance, time zones, cultural differences, and so on. With regard to the latter, most studies compiled multiple indicators to an overall index of virtual work (e.g., Andressen et al., 2012). Other studies experimentally manipulated virtuality, using different levels of media richness (Huang et al., 2010) or specific features of the technology used (e.g., anonymity, Sosik et al., 1999). Third, studies considered virtuality either in relation to the leader (e.g., spatial distance and degree of electronic communication with the leader) or in relation to the team (e.g., team dispersion and degree of electronic communication within the team). While conceptually quite interesting, these diverse operationalizations again make quantitative aggregations difficult, so that we summarize also the findings of cluster 2 descriptively along the introduced leadership taxonomy.

Among the 18 field studies that examined the moderating effects of virtual work, four studies used multiple measurements and multiple data sources, examining task-oriented (i.e., transactional) leadership, relational-oriented (i.e., LMX and empowering) leadership, and change-oriented (i.e., transformational) leadership. Moreover, six experimental studies compared leadership effects under varying degrees of virtual work. With respect to task-related leadership, results showed that transactional leadership was even more effective in highly virtual settings as compared to face-to-face work. This is in line with a cues-filtered-out perspective, suggesting that clear task and goal assignments are particularly important when information exchange is limited (e.g., Konradt et al., 2003). Specifically, two field studies with multi-source and multi-measurement designs found that the relationship between transactional leadership and objective performance was stronger when the spatial distance to the leader was high as compared to low (Howell & Hall-Merenda, 1999; Howell et al., 2005). Similarly, an experimental study with multi-source data and one measurement found that the effects of transactional leadership on creativity were stronger in an anonymous group decision support system as compared to a non-anonymous group decision support system (Sosik et al., 1998; but see also Sosik et al., 1999). Thus, the majority of these findings suggest that behaviours pertaining to transactional leadership, such as goal setting and clear communication of expectations, are even more impactful when the level of virtuality is high rather than low.

With respect to relational-oriented leadership, Hill and Bartol (2016) used a field study with multiple data sources (leader and followers) and two measurement points in time to examine whether empowering leadership was more effective in geographically dispersed teams. Their results indicated that the positive relationship between empowering leadership and team collaboration was stronger at high (vs. low) levels of team geographic dispersion. This result is in line with the notion that participative leadership styles are particularly suited for virtual settings because they provide followers with more latitude to cope with higher self-management demands associated with virtual work (e.g., Hoch & Kozlowski, 2014). Moreover, two field studies examined LMX using multi-source data and two time points. However, the physical distance between leader and followers did not moderate the positive relationship of LMX with objective performance (Howell & Hall-Merenda, 1999); similarly, team geographic dispersion did not moderate the positive relationship between LMX and team decision-making (Gajendran & Joshi, 2012). These results do not support assumptions that might have followed from Goh and Wasko (2012) described in cluster 1, suggesting LMX to be more strongly related to follower reactions in virtual contexts (i.e., computer-simulated environments) than correlations usually observed in non-virtual contexts. Thus, this result illustrates the importance of considering different degrees of virtuality within the same study of leadership effectiveness.

Finally, with respect to change-oriented (i.e., transformational) leadership, the results revealed a more complex pattern. On the one hand, virtual work seems to reduce the connection between transformational leadership and follower reactions. For instance, based on a sample of 109 bank managers and their 317 employees, Howell and Hall-Merenda (1999) examined whether the spatial distance between leader and follower moderated the relationship between transformational leadership and follower performance gathered from company records 18 months later. Results indicated that spatial distance from the leader decreased the positive relationship between transformational leadership and follower performance. In another study, Howell et al. (2005) employed a time lag of 1 year between the measurement of transformational leadership and objective unit performance. Again, results showed that spatial distance to the leader decreased the link between transformational leadership and performance. Thus, although transformational leadership in general also “works” in virtual work settings (see results of cluster 1), spatial distance to the leader seems to diminish the effects of this leadership style. This result is in line with a cues-filtered-out perspective (e.g., Culnan & Markus, 1987), suggesting that electronically mediated communication reduces social cues that are central to transformational behaviours (individual consideration, enthusiastic communication of a clear vision, etc.).

Interestingly, however, experimental studies on moderating effects of virtual work on change-oriented (i.e., transformational) leadership do not support a cues-filtered-out perspective but are rather in line with a deindividuation perspective (Lea & Spears, 1992; Spears & Lea, 1994), suggesting that virtuality increases the effects of transformational leadership behaviours because these emphasize broader social identities. Specifically, Purvanova and Bono (2009) found that transformational leadership effects on team performance were stronger in virtual teams as compared to face-to-face teams. Likewise, Sosik et al. (1998) reported higher positive effects of inspirational leadership (as one aspect of transformational and change-oriented leadership) on creativity in an anonymous as compared to a non-anonymous group decision support system. Moreover, Kahai et al. (2012) compared the influence of transformational and transactional leadership under different communication technologies, i.e., during chat and during a virtual world condition. The results showed transformational leadership to be more effective in the chat as compared to the virtual world condition, suggesting information richness of the communication technology as an important driver of the moderating effect of virtuality (see also Kahai et al., 2003). However, it should be noted that other experimental studies found no moderation of transformational and transactional leadership effects by virtuality at work (Hambley et al., 2007; Hoyt & Blascovich, 2003; Huang et al., 2009; Kahai et al., 2003; Sosik, 1997; Sosik et al., 1998, 1999). Thus, further research is necessary to disentangle these rather complex patterns.

Nevertheless, these initial findings suggest that change-oriented (transformational) leadership effects might indeed be qualified by the level of virtuality, but in different ways. High levels of virtual interaction between leader and followers seem to hamper transformational leadership, which is in line with cues-filtered-out theories (e.g., Daft & Lengel, 1984) arguing that the structural features of virtual work (e.g., high degree of electronic communication) impede the transmission and impact of transformational leadership behaviours (Purvanova & Bono, 2009). In contrast, virtual collaboration among team members seems to facilitate the effects of transformational leadership with respect to team-related processes. The isolation of individual team members might increase their need for orientation and belonging and might therefore support effects of transformational leadership that emphasize the team’s identity and vision, which would be in line with deindividuation accounts (e.g., Lea & Spears, 1992; Spears & Lea, 1994).

Overall, the reviewed studies of cluster 2 suggest that both task-oriented (i.e., transactional) and relational-oriented (i.e., empowering) leadership can be even more effective when virtuality is high. In addition, change-oriented (i.e., transformational) leadership also appears to be even more effective in highly virtual contexts when virtuality refers to the interaction among followers (in teams), but not when virtuality refers to the interaction between leader and followers. However, it should be noted that these initial results are based on quite diverse studies and samples. For instance, the described differences in transformational leadership effects depending on the focus of virtuality (team vs. leader–follower interaction) are confounded with differences in study design (experimental vs. field research) and sampling (students vs. workers as participants), so that it remains unclear whether the observed pattern is driven by virtuality focus or by other aspects. Moreover, the obtained studies on the moderating effects of virtuality on transformational leadership referred to different subdimensions of virtuality, for instance, spatial distance (e.g., Howell & Hall-Merenda, 1999; Howell et al., 2005) or anonymity of followers (Sosik et al., 1998). Depending on the varied subdimensions, however, different moderating processes and follower reactions are conceivable. Thus, in order to better understand and integrate these initial findings, more systematic research is desirable that allows to disentangle the different and partly conflicting influences, and also allows a more differentiated analysis of the different subdimensions of virtuality and their effects on leadership processes. Based on the insights of our current review, we next describe central steps for such future research that might help to move the field forward in this evolving research area.

What we need to do

The importance of leadership in virtual work settings will probably even increase, not only due to ongoing technological advancements and burgeoning distributed work arrangements but also due to external pressures such as climate change, requiring to reduce commuting traffic and the frequency of in-person meetings. The reviewed studies covering highly virtual work contexts only (i.e., cluster 1) suggest that all active leadership types considered are positively correlated with desired follower reactions, which is in line with prior reviews (Brown et al., 2021). The reviewed studies examining leadership effects as a function of virtuality (i.e., cluster 2), however, revealed a more complex pattern of results. For example, change-oriented (i.e., transformational) leadership appears to be even more effective in highly virtual work contexts when virtuality refers to the interaction among followers, whereas it seems to be less effective when virtuality refers to the interaction between leader and follower. Other studies on transformational leadership, however, did not report a moderating effect of virtuality. The virtuality-as-a-paradox perspective (Purvanova & Kenda, 2018) is well suited as an overarching view for these complex results. This perspective advocates a balanced view on virtuality’s risks and opportunities, instead of exclusively focusing on one or the other. That is, this view simultaneously considers virtuality as a negative and positive force where the bright or the dark side may prevail or cancel each other out (Purvanova & Kenda, 2022). We believe that virtuality-as-a-paradox is a promising overarching view on leadership in virtual work contexts and deserves further theoretical development. Below, we take an important step towards building a theory that can help better disentangle the paradoxical effects by providing a parsimonious but yet inclusive conceptualization of leadership in virtual work settings.

Relatedly, our review has revealed quite diverse conceptual and methodological approaches in this interdisciplinary field, making the aggregation and integration of extant findings difficult. Moreover, the methodological approaches of many pioneering studies (e.g., cross-sectional designs, self-reported data only, only one level of virtuality) limit the generalizability of findings, requiring replication and extension to more mature stages of this evolving field. To consolidate this field further, we suggest developments in three major areas, addressing both conceptual and methodological issues but also the consideration of recent technological developments.

Standardize conceptualization and measurement of virtual work settings

One main observation throughout our review is the high variance and lack of consensus on how virtual work was conceptualized and measured (see also Raghuram et al., 2019; Schweitzer & Duxbury, 2010). In fact, we had difficulties finding even two studies with the same conceptualization of virtual work. In addition, many studies provided only cursory descriptions of their settings and/or measurement, particularly studies of cluster 1 which did not consider virtual work as a moderator variable. In the light of these construct validity problems, future research would strongly benefit from a more consensual understanding of virtual work settings, both with respect to theoretical integration and measurement.

Instead of rather gross descriptions of virtual work settings as “virtual” or “remote” work, time is ripe to be more specific when considering work settings in terms of level and features of virtuality. In addition to a more precise assessment of this work characteristic, the various features of virtual work have specific and partly diverse implications for psychological processes of leadership effects. For instance, while spatial dispersion should hamper transformational leadership due to reduced opportunities to communicate an inspiring and motivating vision, synchronous communication tools (e.g., chat or video-conferencing) have the potential to meet quickly and spontaneously with followers (also compared to traditional work contexts), which should positively contribute to individual consideration of followers. Analyses that neglect such subdimensions of virtual work might overlook important but opposing mechanisms (e.g., Gajendran & Joshi, 2012; Hill et al., 2014). Thus, considering core elements and related facets of virtuality holds the potential for an improved understanding of paradoxical effects of leadership in virtual work contexts. In line with a virtuality-as-paradox perspective (Purvanova & Kenda, 2022), a more fine-grained approach is needed to understand when virtuality’s positive or negative effects prevail, and when opposing forces block each other.

Figure 3 depicts our approach to leadership in virtual settings (LiVS). We consider broad leadership categories (i.e., task-, relational-, and change-oriented) and global follower reactions (i.e., job performance and attitudes) to illustrate the influence that leadership has on followers. We further propose that the virtuality of the work context moderates leadership effects. We suggest the relative amount of electronically mediated leader-follower interaction as the core element of leadership in virtual work settings – which is in line with the definition of e-leadership (Avolio et al., 2014) – and suggest measuring this core element as the number of mediated communication acts in relation to overall communication acts. This conceptualization is slightly different from extant approaches that suggest multiple dimensions as defining elements of virtual work, including spatial dispersion or cultural heterogeneity (e.g., Hoch & Kozlowski, 2014; Kirkman & Mathieu, 2005; Raghuram et al., 2019). However, virtual work is also possible (and prevalent) when persons work in the same location (e.g., building and office) and belong to the same cultural group. Therefore, these features are not necessary conditions of virtual work. Instead, we concentrate on electronic mediation of information and collaboration as the core element of this work form. Such a focused conceptualization enables a more precise measurement and better theoretical connection with psychological process theories on leadership and followers’ experience and behaviour, such as cues-filtered-out approaches (e.g., Culnan & Markus, 1987) and deindividuation theories (Lea & Spears, 1992; Spears & Lea, 1994).

Figure 3. Model on leadership in virtual settings (LiVS model).

However, in order to provide not only a parsimonious but also an inclusive conceptualization of virtual work as a context of leadership, we consider other dimensions that have been suggested as features of virtual work as qualifying factors of electronically mediated interaction (see Figure 3). For example, spatial dispersion as perhaps the most often mentioned aspect besides electronic mediation of interactions (e.g., Raghuram et al., 2019) describes the relative geographical distance, for instance, between leaders and followers (e.g., O’Leary & Cummings, 2007; Schweitzer & Duxbury, 2010). While high spatial dispersion can additionally increase negative effects of electronic mediation of leader–follower interaction (e.g., lack of context information, feelings of isolation), it can also increase positive effects of electronic communication (quick access) when leaders have to stay connected with followers across higher distance, for instance in sales or customer service industries (e.g., O’Leary & Cummings, 2007).

Temporal dispersion (or asynchronicity of interaction) describes the relative degree to which interaction happens at the same time and has also been frequently mentioned as a core feature of virtuality (e.g., Kirkman & Mathieu, 2005; O’Leary & Cummings, 2007; Schweitzer & Duxbury, 2010). As a moderator in our conceptualization, temporal dispersion can increase the negative effects of electronic mediation (e.g., misunderstandings due to lack of feedback cues), but offers also positive effects by providing additional latitude for followers’ time and stress management (e.g., Dennis & Valacich, 1999; Dennis et al., 2008).

Information richness, i.e., technologies’ capacity to convey physical, contextual, and social-emotional cues (e.g., Daft & Lengel, 1984, 1986), is another important moderator of electronic mediation effects, particularly for leadership in virtual settings. Indeed, our review has revealed a wide variety of technological devices used in the obtained studies, ranging from rather lean media (e.g., only text-based, such as online chat) to quite complex technologies (e.g., video and audio information, such as video-conferencing) that come close to the information richness of unmediated face-to-face interaction. Low levels of information richness further increase negative effects of electronic mediation of interaction, particularly those described by cues-filtered-out approaches (e.g., Daft & Lengel, 1984, 1986), whereas high levels of information richness should rather reduce such negative effects.

Reprocessability of interactions, i.e., the documentation and accessibility of interaction information (e.g., Breuer et al., 2016), is another feature that we consider as an important moderator of electronic mediation of leadership interaction at work. Reprocessability of interaction facilitates fairness and trust in interactions (e.g., Colquitt & Zipay, 2015; Mayer et al., 1995) and thus might counteract potential conflicts and distrust that can arise from the lack of information and context cues in virtual interaction. For instance, reprocessability of interactions has been shown to qualify negative effects of virtuality on trust in virtual teams (Breuer et al., 2016). At the same time, reprocessability of information (e.g., recorded online meetings) is easier to realize in virtual as compared to face-to-face settings, simply because electronic mediation facilitates automatic storage and documentation.

While our model focuses on leader behaviours and interactions between leaders and followers, processes among followers can also be considered. As described for transformational leadership, virtuality between leaders and followers seems to have different consequences than virtuality among team members. Thus, in addition to the interaction between leaders and followers, technology features can also qualify the working conditions of the followers, leading to different requirements and needs that leaders want to consider in their behaviours. For instance, teams with high levels of virtuality not only need more task- and relational-oriented leadership as compared to teams with low levels of virtuality (see also Brown et al., 2021) but seem also to benefit more from change-oriented leadership (e.g., Purvanova & Bono, 2009). Notably, these changing requirements within a team differ from the qualifying effects of technology on the more direct interaction between leaders and followers. In general, this differentiation further illustrates the importance of a process-oriented analysis of technological effects in work contexts.

The described qualifiers of leadership in virtual work are considered as initial examples but not as an exhaustive list. Further potential moderators are the diversity of followers (e.g., Horwitz & Horwitz, 2007; Miller et al., 1998) but also the diversity of technologies used (e.g., Goodhue & Thompson, 1995). In addition, current and future technological advancements might be integrated in these moderators, for instance, immersion as part of information richness that can be even higher in virtual settings (virtual reality) as compared to unmediated face-to-face settings (e.g., Höddinghaus & Hertel, 2019). Moreover, the interaction between the different moderators might explain the additional variance. For instance, high information richness and low temporal dispersion (i.e., synchronicity) of information exchange might compensate for challenges associated with spatial dispersion (e.g., lack of a shared context). In order to structure these different influences, our suggested framework with one main process and multiple moderating conditions is well suited to integrate such potential future developments. Thereby, our model provides an overarching framework to disentangle virtuality’s features and to better understand when positive or negative aspects prevail and when they cancel each other out.

Finally, in addition to a more structured analysis of virtual work settings, we also support a more structured analysis of leadership in this context. Echoing the recent discussions and requests for more behaviour-oriented leadership research instead of relying on rather gross behaviour clusters (i.e., leadership styles) that are partly overlapping with other concepts (e.g., Hoch et al., 2018; Shaffer et al., 2016), we would wish that future research also in the context of virtual work follows a more specific but also theory-based approach (see Van Quaquebeke & Felps, 2018, for a more elaborate discussion of this issue).

Solid methods and study designs

In addition to conceptual issues, our review also revealed quite different methodological approaches in the obtained studies that made a quantitative aggregation difficult. This diversity, partly due to interdisciplinary work and different research traditions in this emerging field, was certainly fruitful in the initial phases of this emerging field. However, in order to consolidate this research, more standardization and methodological rigour are desirable if not necessary (e.g., Antonakis et al., 2010; Lonati et al., 2018). Below, we note three broader demands that we would consider central for future studies on leadership in virtual work settings:

• Exceed cross-sectional single source self-report designs whenever possible. The majority of studies in the current review are plagued with rather high risks of common method biases (Podsakoff et al., 2012). While this might be fine for pioneering stages of research, more rigorous and informative designs are desirable at maturing stages. Given that virtual settings are technically well equipped to collect longitudinal data, leadership researchers might find even better conditions for such endeavours in virtual work settings than in more traditional settings. Moreover, virtual work settings might also provide better conditions to collect non-reactive data, such as performance data but also process data such as communication behaviour (Guo et al., 2017), eye movement (e.g., Gerpott et al., 2018), or even skin conductance (e.g., Akinola, 2010).

• Include different levels of virtuality, with precise descriptions of the different levels (technologies used, etc.) to enable moderator analyses and to address endogeneity problems (e.g., Antonakis et al., 2010). As shown in our review, many of the obtained studies were mute with respect to the question of whether leadership in virtual settings is different from leadership in more traditional settings. This issue is not only important to connect research on leadership in virtual settings with extant leadership approaches but also to enable more specific recommendations if and how leadership might have to be adapted to virtual work. Moreover, our review revealed a considerable amount of mixed findings that might be due to neglected variables. Future research might consider such moderating factors more explicitly.

• Consider evidence from different types of studies (triangulation). While both experimental and field studies exist on leadership in virtual work settings, our review reveals that multi-study papers are still quite rare. At the same time, type of study (experimental vs. field) was confounded with the sampling of studies (students vs. workers as participants) among other factors (i.e., time scope of collaboration, meaningfulness of the task, etc.). In order to combine the benefits of experimental and field study approaches, i.e., high internal and high external validity, multi-study papers are highly warranted.

Consider technological advancements

The studies included in our review mainly focused on technologies such as email and telephone. We think this is warranted because email and telephone are highly prevalent in daily communications. However, communication technology is developing at a fast pace and provides multiple tools that still await scientific elaboration, such as video conferencing (e.g., GoToMeeting, MS Teams, Zoom), collaborative software (e.g., Slack), or enterprise social networking services (e.g., Yammer, Jive), and more recent advancements, such as augmented reality and virtual reality applications (e.g., Bacca et al., 2014; Bailenson, 2018; Srivastava & Chandra, 2018). Interestingly, a high immersion and a high sense of social presence in virtual reality meetings might produce even higher engagement and commitment than unmediated leader–follower interactions (e.g., Höddinghaus & Hertel, 2019, for an initial study).

Moreover, leadership research has only begun to embrace automation of leadership (e.g., Van Quaquebeke & Gerpott, 2023; Wesche & Sonderegger, 2019). Due to technological progress, sophisticated technologies based on artificial intelligence and big data are able to support humans in the analysis and interpretation of complex data such as corporate performance data or medical data (Haenssle et al., 2018; Richards et al., 2019). Thus, technologies are transcending beyond their original function as pure tools into a role in which they perform leadership functions such as the allocation of tasks and resources, appraisal of performance, planning of shifts, and compensation (Harms & Han, 2019; Wesche & Sonderegger, 2019). Studies addressing automated leadership both theoretically and empirically are desirable to provide guidelines (and limitations) for their potential implementation and development. The design of automated leadership systems should not only follow organizational goals (e.g., system performance) but also be human-centred and pursue the interests of employees (e.g., health and well-being, usability, and ethical standards).

Apart from beneficial and liberating opportunities, many features of digital collaboration technologies have considerable potential for misuse, which have to be better understood as well. Permanent availability, blending of personal and professional life, biased decision algorithms, and the replacement of human workers due to automation are just a few examples of potential consequences that probably cause more stress than release for employees. So far, only few studies explicitly considered such negative outcomes of leadership in virtual work settings (see Hite, 2009; Konradt et al., 2003, for initial examples). Scholars might maintain a healthy scepticism towards automated adoptions of new technologies. Moreover, work and organizational psychology offers many starting points for human-centred technology design (e.g., Hertel et al., 2017; Van Quaquebeke & Gerpott, 2023), starting from leader and follower needs instead of technological advances.

Finally, a perspective that has also been neglected is the implications of virtuality (level) on the selection of certain leadership types or specific behaviours. For instance, external pressure to change to virtual work, for instance, during the COVID-19 pandemic, might have led to more passive or even destructive leadership (e.g., Rudolph et al., 2021). Thus, a fruitful extension of our suggested framework (Figure 3) might be to also consider influences of virtuality on the selection of leadership behaviours, which might also be moderated by the different subdimensions of virtual work.

Practical implications

Returning to Natalie and her situation described in the Introduction, a central goal of this review was to examine which evidence-based recommendations for leaders in virtual work settings might be possible based on the current state of the literature. Apart from the described limitations due to a high diversity of the literature, we believe that initial recommendations are nevertheless possible, particularly based on studies with multi-source data and multi-measurement designs. First, our findings suggest that supervisors in virtual settings are well advised to engage in both task-oriented and relational-oriented leadership. This recommendation is not only in line with data from virtual settings only (see also Brown et al., 2021), but also with research that compared different levels of virtuality, showing that the effects of these two leadership types even increase with higher levels of virtuality. Thus, virtual settings seem to increase followers’ need for coordination, goal setting, and clear role assignments, but at the same time, feelings of isolation, disconnectedness, and lack of socio-emotional cues seem to increase followers’ relational-oriented needs. Moreover, higher self-management of followers requires participative and empowering leadership behaviours that support followers’ autonomy during virtual work. Finally, the results also suggest that virtual work might increase team-oriented aspects of change-oriented leadership but reduce aspects directed towards supporting individual followers.

Second, our review also underlines that leaders need to understand virtual work in a more differentiated way, considering not only the amount of electronic mediation but also the qualifying subdimensions of spatial dispersion, temporal dispersion, information richness, and reprocessing ability of information, among others. Depending on the combination of these factors, different challenges (e.g., followers’ feelings of isolation and lack of support) but also benefits (e.g., more latitude and more spontaneous contact) are conceivable and have already been initially found in extant research. In addition to further research in this respect, validated training concepts are highly needed that prepare and support leaders to cope with additional demands and partly changing the needs of their followers in increasingly virtual work settings.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The work was supported by the Deutsche Forschungsgemeinschaft [1712/2].

Notes

1. We included Sosik et al. (1997) and Sosik et al. (1999) even though the articles used the same data. However, in their analyses the studies focused either on the individual level or the team level of the data, providing unique results.