The effects of inclusion and overinclusion: explanations for treatment matter

ABSTRACT

The current work quantitatively explored whether the explanation for overinclusion moderates its effect. In Study 1, female participants were excluded, included, or overincluded in a virtual ball-tossing game and given reason to believe this treatment was due to their gender or not. In Study 2, female participants imagined a scenario in which they were included or overincluded due to their gender or not. The effect of inclusion versus overinclusion on participants’ need satisfaction was significantly moderated by the explanation for this treatment in both studies. This work suggests that including or even overincluding members of traditionally underrepresented groups is not sufficient for members of these communities to have positive group experiences; the explanations they have for their treatment matters.

KEYWORDS:

• Overinclusion
• inclusion
• minority spotlight effect
• gender

When someone is overincluded they are included in that they are acknowledged and accepted, but their experience goes beyond traditional inclusion because they receive a disproportionate share of attention relative to the other people in the group (De Waal-Andrews & van Beest, 2020). Common examples of overinclusion include receiving recognition in a group for special occasions like birthdays or accomplishments such as receiving an award. However, someone may be overincluded ‘simply because their behavior or spirit makes them the center of attention’ (p. 107, De Waal-Andrews & van Beest, 2020). Although not explicitly stated as such, these examples and descriptions paint a generally positive picture of overinclusion experiences.

Other instances of overinclusion may not be experienced positively. For example, imagine being the only person of color or woman in an engineering class. People may receive a disproportionate amount of attention in a group by virtue of holding underrepresented social identities and feel the unwelcome burden to serve as representatives for these social identities (Crosby et al., 2014). Work on affirmative action also suggests there may be limits to the positivity of overinclusion (Turner & Pratkanis, 1994). Importantly, affirmative action is not overinclusion. Instead, affirmative action is a set of policies intended to improve equity and diversity in the workforce in which companies must make concerted efforts to consider similarly qualified members of minoritized groups for employment opportunities. However, affirmative action is often misunderstood as inclusion or overinclusion solely because of one’s group membership and thus attracts extra attention to minoritized groups. When affirmative action is misunderstood in this way it lowers women and people of color’s self-evaluations of ability.

Although anecdotally one may find it easy to think of real-life examples of overinclusion, relatively little quantitative psychological work directly investigates overinclusion. The majority of the work that studies overinclusion as its own independent construct is rooted in the social exclusion literature. The social exclusion literature demonstrates the effects of overinclusion as a construct in and of itself in part by precisely comparing its experience to that of inclusion. The purpose of the current work was to focus on the experience of overinclusion specifically, its comparison to inclusion, and whether the explanation for this treatment matters. As such, the current work is most firmly rooted in the social exclusion literature. However, this work bridges the gap between the social exclusion literature’s investigation of overinclusion and other literatures, as the explanation of interest investigated is whether one attributes their treatment to their gender identity.

Emblematic of the ironic relative lack of research on overinclusion, the majority of social exclusion research does not actually investigate overinclusion. Instead, the vast majority of research studies inclusion and uses it primarily as a comparison condition for exclusion. A wealth of research demonstrates that inclusion is preferable to exclusion (e.g., Ren et al., 2017). These effects are consistent with theories on belonging and evolution which suggest that the need to belong is powerful, perhaps in part due to the fact that belonging in groups was essential for early human survival (Baumeister & Leary, 1995; Williams, 2009). The dominant paradigm in this inclusion and exclusion work is Cyberball, the most common version of which involves participants playing a virtual ball-tossing game with two other people, who unbeknownst to participants are actually computer programmed confederates. When participants are included, the confederates toss participants the ball a third of the time, and when participants are excluded, the confederates typically toss the ball to participants once (to demonstrate that it is possible) and then never again. After the game, participants typically self-report their mood and need satisfaction. Need satisfaction is a composite of four fundamental human needs: the need to belong, the need for control, the need for self-esteem, and the need for a meaningful existence. Exclusion consistently reduces need satisfaction in comparison to inclusion both immediately (reflexively) and after participants are given time to process their experience (reflectively) (e.g., Ren et al., 2017; Williams, 2007). Exclusion can also increase negative mood relative to inclusion, although mood effects are less consistent and can depend on paradigm (Bernstein & Claypool, 2012).

Although overinclusion is paradoxically understudied, some researchers have proposed that overinclusion is preferable to inclusion as a comparison condition for exclusion effects (e.g., Meneguzzo et al., 2020), arguing that overinclusion and exclusion similarly violate expectations of fair play (Kawamoto et al., 2012) or result in people feeling similarly conspicuous and knowledgeable of other people’s responses to them (Williams et al., 2000). Consistent with this argument, included participants and those in neutral control conditions experience similar need satisfaction and mood, suggesting that inclusion is a default, expected state given people’s common experiences of interacting with those that accept them (Dvir et al., 2019). The research comparing exclusion and overinclusion generally uses Cyberball or variations of Cyberball which operationalize overinclusion as participants receiving all or the clear majority of the other supposed players’ passes. This work finds excluded and overincluded participants feel similarly different from others (De Waal-Andrews & van Beest, 2020). Neuroscience work has also found that there are regions of the brain such as the dorsal anterior cingulate cortex (dACC) that are similarly activated under exclusion and overinclusion but not under inclusion, supporting the assertion that overinclusion and exclusion involve similar expectation violations (Cheng et al., 2020). However, importantly these feelings of expectancy violation or conspicuousness aren’t necessarily or equally negative. Research that compares overinclusion to exclusion using Cyberball as well as less common exclusion paradigms generally finds that exclusion elicits lower need satisfaction and less positive mood than overinclusion (Carter-Sowell et al., 2010; De Waal-Andrews & van Beest, 2020; Erel et al., 2021; Kawamoto et al., 2012; Kwok et al., 2018; van Beest et al., 2011; Venturini et al., 2016; Wolf et al., 2015). In fact, people also experience greater need satisfaction when their overinclusion comes with a monetary cost than when their exclusion comes with a monetary cost, suggesting attention is preferable even when it is punitive (van Beest & Williams, 2006). Indeed, these findings are consistent with theories regarding the important evolutionarily adaptive properties of inclusion (Baumeister & Leary, 1995; Williams, 2009).

Although comparisons between the effects of inclusion and overinclusion are rarely the conceptual focus of research, some work does compare these two conditions. Some of this work generally finds that inclusion and overinclusion are similar with regard to need satisfaction and mood in both Cyberball and other exclusion paradigms (Carter-Sowell et al., 2010 Study 3; Kawamoto et al., 2012; Williams et al., 2000; Wolf et al., 2015). Or in some cases, such as in a study on how attributing ostracism to racism hinders recovery, the overinclusion condition was so subtle that it was not detected and the inclusion and overinclusion conditions were collapsed (Goodwin et al., 2010). Work on overinclusion that occurs progressively over multiple rounds of Cyberball also generally finds no effects of overinclusion over time on self-reported need satisfaction and moods (Ho et al., 2014). And although some studies find differences between overinclusion and inclusion with regard to need satisfaction, they may find similar mood effects (van Beest et al., 2011; Venturini et al., 2016).

Other research suggests that overinclusion is preferable to inclusion. For example, in one classic Cyberball study participants felt greater need satisfaction when overincluded than when included (van Beest et al., 2011). In another classic Cyberball study, overincluded participants reported increased control, belonging, and meaningful existence compared to included participants (Niedeggen et al., 2014). However, these participants did not differ in terms of self-esteem. People with excessive acquisition tendencies have also reported greater need satisfaction following overinclusion in Cyberball than inclusion (Kwok et al., 2018). Some studies that have implemented adapted Cyberball paradigms have also found evidence that overinclusion can be preferable to inclusion (Carter-Sowell et al., 2010; Erel et al., 2021; Venturini et al., 2016).

Although the bulk of the research thus far suggests that overinclusion is generally as or more positive than inclusion, there are some indications that this is not always so. For example, in a Cyberball game persistence in the game was greater amongst included than excluded participants, however overincluded and excluded participants persisted at similarly low rates (Williams et al., 2000). In addition, when in a modified Cyberball game in which receiving ball tosses comes at a financial cost, overinclusion is experienced more negatively than inclusion (van Beest & Williams, 2006). And in Cyberbomb where the ‘ball’ being tossed is instead a bomb that is a threat to survival in the game, need satisfaction is higher for included than overincluded participants (van Beest et al., 2011). In addition, people retaliate more following overinclusion than inclusion in Cyberbomb by assigning their former group member who indicated a distaste for spicy food more hot sauce to eat (van Beest et al., 2011).

There is also research outside of the traditional exclusion literature that suggests that overinclusion may not always be a positive experience. Researchers have documented the minority spotlight effect wherein members of underrepresented groups are likely to be disproportionally attended to when issues pertaining to their groups are mentioned, recognize this overinclusion, and experience negative emotions and the burden to be the representative for their group (Crosby et al., 2014; Zou & Cheryan, 2015). In support of this hypothesis, female and Black students, when listening to an argument about affirmative action, felt they were looked at and were more ‘in the spotlight’ than their other group members and reported more negative mood than when listening to a control argument not relevant to marginalized groups (Crosby et al., 2014). Interviews with faculty of color also suggest that being under-represented can elicit feelings of hypervisibility and the pressure to serve as a representative for the larger underrepresented group (Settles et al., 2019). Paradoxically, this treatment can also lead people to feel both hypervisible and invisible, as they may not be acknowledged for aspects outside of their social identity and may be excluded from certain opportunities. These findings are also consistent with research demonstrating that misunderstandings of affirmative action as overinclusion on the basis of one’s group membership can have negative effects including decreasing women and people of color’s self-evaluations of ability (Turner & Pratkanis, 1994). These negative effects of affirmative action are eliminated when explicit unambiguous evidence of qualifications is provided

The mixed findings regarding the effects of overinclusion relative to inclusion and the indications that overinclusion may not always be a positive experience suggest that not only is overinclusion an important phenomenon to study in its own right, but that there are likely to be moderators of overinclusion experiences. Indeed, even though inclusion is pretty generally demonstrated and assumed to be preferable to exclusion experiences, there are moderators of inclusion experiences. For example, the effects of being included on information can depend on whether that information is positive or negative (McCarty et al., 2018). In addition, the experience of inclusion depends on how it was obtained, with inclusion granted by others being experienced more positively in terms of need satisfaction and mood than inclusion that is claimed by oneself, presumably as granted inclusion reflects the fact that others see one positively (de Waal-Andrews & van Beest, 2012). Overinclusion also leads to more need satisfaction and positive mood when it was granted by others than when it was claimed by oneself (De Waal-Andrews & van Beest, 2020).

The present study

The current work extends past research by focusing intentionally on the comparison between inclusion and overinclusion and quantitatively investigating a novel moderator that may help shed light on past mixed findings. We propose that the explanation for inclusion and overinclusion moderates these effects, perhaps even preventing these experiences from being as positive. Given that the limited prior research on the negative effects of inclusion and overinclusion generally focuses on experiences of tokenness and being reduced to a representative of an underrepresented group (e.g., Crosby et al., 2014; Zou & Cheryan, 2015), the current work specifically focuses on the influence of reasons to believe that one is only being included due to one’s demographic characteristics, in this case gender. Our hypothesis was that the experiences of inclusion vs. overinclusion in terms of the classic outcome variables from the social exclusion literature of need satisfaction and mood (Williams, 2009) would be moderated by whether or not participants were given reason to believe their treatment was due to their gender. Differences between overinclusion and inclusion may emerge when participants are not given reason to think their treatment is due to their gender. Although the prior research is mixed, in Cyberball paradigms overinclusion often leads to more need satisfaction and positive mood than inclusion. However, we expected that any differences between the experiences of inclusion and overinclusion would be eliminated when participants would have reason to think their treatment was due to their gender, as prior research suggests that being included but reduced to one’s group membership can be an aversive experience (Crosby et al., 2014; Settles et al., 2019). This prediction is also consistent with prior work suggesting that the positive effects of inclusion and overinclusion are attenuated when these experiences aren’t granted by others, and thus are not potential reflections of others’ positive attitudes (de Waal-Andrews & van Beest, 2012; De Waal-Andrews & van Beest, 2020).

This hypothesis was tested in studies utilizing two different paradigms. In Study 1, female participants were included, excluded, or overincluded in a Cyberball game and given reason to believe this treatment was due to their gender or not. In Study 2 female participants imagined being in a scenario in which they were included or overincluded due either to their gender or their personality. In both studies, participants completed self-report measures of their fundamental need satisfaction and mood.

Study 1

Study 1 tested our key hypothesis using the classic Cyberball paradigm.

Methods

Participants and design

Participants were 323 female undergraduate students who participated in exchange for partial course credit. Four participants were excluded from analyses because they were under 18 years of age. Additionally, participants were excluded from analyses if they requested we discard their data during the debriefing process (n = 8), if they reported having already participated in a study which used Cyberball (n = 16), or if their response was associated with the second instance of a repeated IP address in the dataset, indicating someone using this device had already participated in this Cyberball study (n = 6). Thus, our final sample consisted of 289 participants (M_age = 18.77). The majority of participants were Caucasian (n = 203), with others indicating Asian/Pacific Islander (n = 47), Hispanic (n = 10), other identifications (n = 27), or that they preferred not to respond (n = 2). Participants were randomly assigned to one condition of a 3 (Treatment Condition: Exclusion vs. Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) between-subjects design. Sample size was determined by ensuring at least 40 participants per cell, and then recruiting more than 240 participants in total in anticipation of needing to exclude some participants. This study was approved by the Purdue University Institutional Review Board (#1201011749) and was not preregistered.

Procedure

Participants were recruited via the online recruitment system SONA. Only participants who completed a screening survey at the beginning of the semester and selected their gender as female were able to see this study in the list of potential studies to complete. Participants completed the study online via Qualtrics.

After consenting to participate, participants were told they would play a mental visualization game with two other people from across campus. First, they were asked to type in their name and they were told that it would show up during the game. This was done so that it would make sense when participants saw the other players’ names (which would both be male). Participants were randomly assigned to one of two explanation conditions where they were told that the purpose of the study was either to investigate mental visualization (control explanation) or to investigate women’s mental visualization specifically (gender explanation). For those in the control explanation condition, their instructions read: ‘For this game, we are particularly interested in mental visualization in general, as very little is known about this. The majority of research that has been conducted in the past has focused on actual visualization. Therefore, in this game, our primary focus is on mental visualization.’ For those in the gender explanation condition, their instructions read: ‘For this game, we are particularly interested in female participants’ ability to mentally visualize, as very little is known about this. The majority of research that has been conducted in the past has been conducted using male participants. Therefore, in this game, our primary focus is on women’s mental visualization.’

Next, participants read instructions about the Cyberball game and played Cyberball. Cyberball is a virtual ball-tossing game that is often used to study exclusion (Williams, 2009). Participants believe they are throwing a virtual ball back and forth with other players (in this case, two other players); however, in reality, the game is pre-programmed for the participant to receive the ball a certain amount of the time. In this study, participants were randomly assigned to be either overincluded, included, or excluded. Those who were in the overinclusion condition received all but two ball tosses from the other players. In the inclusion condition, participants received the ball a third of the time. Lastly, in the exclusion condition, participants received the ball once near the beginning of the game and then never again.

After the game, participants completed measures to assess their need satisfaction and mood twice as well as manipulation checks. For questions about need satisfaction and mood (Williams, 2009), the first-time participants responded about how they felt during the game (or reflexively), and the second time (a few minutes later), participants responded about how they felt in the current moment (reflectively). The second set of questions are meant to assess recovery of need satisfaction and mood. Twelve items were used to assess need satisfaction capturing the four fundamental needs of belonging (e.g., ‘I felt/feel disconnected’), self-esteem (e.g., ‘I felt/feel good about myself’), control (e.g., ‘I felt/feel powerful’), and meaningful existence (e.g., ‘I felt/feel meaningless’) answered on a scale from 1 (not at all) to 7 (extremely). These items were reverse-coded when necessary and combined to form a single measure of need satisfaction (reflexive α = .94; reflective α = .92), as suggested by Williams (2009) and reported in other work (e.g., Jones & Kelly, 2010), with higher numbers indicating more satisfaction. Mood was assessed with 9 items (e.g., ‘I felt/feel positive’) answered on a scale from 1 (not at all) to 7 (extremely). These items were reverse-coded when necessary and combined to form a single measure of mood with higher numbers indicating a more positive mood (reflexive α = .90; reflective α = .89).

Participants responded to a manipulation check about the Cyberball game to determine whether participants accurately interpreted the treatment condition. This question read: ‘Please estimate the percentage of ball tosses you received in the game. Note that in a three-person game if everyone received the ball an equal amount of times this would come to around 33%.’ The responses for this item were on a sliding scale from 0 to 100%.

Finally, participants responded to a manipulation check for the explanation manipulation. This question asked participants ‘At the beginning of the experimental session today, what did the instructions say the experiment focused on?’ with response options of ‘women’s mental visualization,’ or ‘mental visualization in general.’

Results

Explanation manipulation check

Responses to the explanation manipulation check were significantly associated with explanation condition, X²(1, N = 289) = 81.66, p < .001. A 64% of participants in the gender explanation condition and 87% of participants in the control condition correctly identified whether or not the instructions said the study was particularly interested in women’s mental visualization. The below analyses were conducted as intended with those participants who failed the explanation manipulation check removed. Although a priori power analyses were not conducted, sensitivity analyses were conducted using G*power (Faul et al., 2007). Sensitivity analyses for between factors effects in repeated measures ANOVAs were conducted as the key hypothesized effect did not involve the within-subjects factor. Given this final sample, effects of Cohen’s f = 0.23 could be detected with 80% power.

Treatment manipulation check

The treatment manipulation check was analyzed with a 3 (Treatment Condition: Exclusion vs. Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) between-subjects ANOVA. Only a significant main effect of treatment condition emerged, F(2, 212) = 185.63, p < .001, η_p²=.64. Overincluded participants estimated receiving the greatest percentage of ball tosses (M = 45.07, SD = 16.03), followed by included participants (M = 31.18, SD = 10.48), and then excluded participants (M = 6.81, SD = 7.50), with follow-up Tukey tests indicating significant differences between each (ps<.001). The main effects of explanation condition, F(1, 212) = 0.06, p=.810, η_p²=.00, and the interaction, F(2, 212) = 0.23, p=.796, η_p²=.00, were not significant.

Primary analyses

Primary analyses consisted of 3 (Treatment Condition: Exclusion vs. Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) × 2 (Timing: Reflexive vs. Reflective) mixed-factor ANOVAs on need satisfaction and mood. Significant main effects of treatment condition were followed up with Tukey tests. For the sake of concision, timing effects are not discussed in the main text, as the focus of this manuscript is on the between-subject variables¹.

Need Satisfaction

A significant main effect of treatment condition emerged, F(2, 212) = 110.64, p < .001, η_p²=.51. Overincluded participants (M = 5.23, SD=.91) and included participants (M = 4.95, SD=.94) reported greater need satisfaction than excluded participants (M = 3.02, SD = 1.01), ps<.001. Overincluded participants and included participants reported similar need satisfaction, p=.203. No significant main effect of explanation condition emerged, F(1, 212)=.22, p=.638, η_p²=.00.

Of primary interest, the treatment condition by explanation condition interaction did not reach statistical significance, F(2, 212) = 2.81, p=.062, η_p²=.03. However, as the descriptive pattern is in line with our hypotheses (see Figure 1), we investigated the post-hoc effects for exploratory purposes. A significant effect of treatment emerged among those in the control condition, F(2, 213) = 84.26, p < .001, η_p²=.44. In the control condition, overincluded participants and included participants reported greater need satisfaction than excluded participants, ps<.001. Overincluded participants also reported greater need satisfaction than included participants, p=.028. A significant effect of treatment also emerged among those in the gender explanation condition, F(2, 213) = 35.24, p < .001, η_p²=.29. In the gender explanation condition, again, overincluded participants and included participants reported greater need satisfaction than excluded participants, ps<.001. However, unlike in the control condition, in the gender explanation condition overincluded and included participants reported similar need satisfaction, p=.724.

Figure 1. The effects of treatment and explanation conditions on need satisfaction in Study 1.

Error bars represent one standard error.

Mood

A significant main effect of treatment condition emerged, F(2, 213) = 38.54, p < .001, η_p²=.27. Overincluded participants (M = 5.79, SD=.91) and included participants (M = 5.60, SD = 1.01) reported greater positive mood than excluded participants (M = 4.37, SD = 1.16), ps<.001. Overincluded and included participants did not differ, p=.738. No significant main effect of explanation condition emerged, F(1, 213) = 1.85, p=.175, η_p²=.01. The treatment condition by explanation condition interaction also did not reach statistical significance, F(2, 213) = 1.47, p=.233, η_p²=.01.

Analyses using complete sample

We report our analyses above with those who failed the explanation manipulation check removed as originally intended. However, a limitation of these analyses is the number of participants who failed the explanation manipulation check as well as the fact that those who failed this manipulation check were not equally distributed across explanation conditions. Thus, we also conducted the analyses retaining those who failed the explanation manipulation check below. Given this sample, the effects of Cohen’s f = 0.20 could be detected with 80% power.

The analyses of the complete sample are similar to those using the sample in which participants who failed the explanation manipulation check are removed with two exceptions. First, consistent with our hypothesis, a treatment condition by explanation condition interaction on need satisfaction reached conventional levels of statistical significance using the entire sample, F(2, 282) = 4.41, p=.013, η_p²=.03. The post-hoc patterns are similar to those using the sample without participants who failed the explanation manipulation check. In the control condition, overincluded participants and included participants reported greater need satisfaction than excluded participants. In the control condition, overincluded participants also reported greater need satisfaction than included participants. In the gender explanation condition, again, overincluded participants and included participants reported greater need satisfaction than excluded participants. However, unlike in the control condition, in the gender explanation condition overincluded and included participants reported similar need satisfaction.

Second, the treatment condition by explanation condition interaction on positive mood also emerged in the complete sample, F(2, 283) = 3.19, p=.043, η_p²=.02. In the control condition, overincluded participants and included participants reported more positive mood than excluded participants. In the control condition, overincluded and included participants reported similar moods. The pattern in the gender explanation condition was similar to the control condition, although less pronounced. Again, overincluded participants and included participants reported a more positive mood than excluded participants. Overincluded and included participants reported similar moods. Complete reports of the analyses using the complete sample can be found here: [https://osf.io/ebq2k/?view_only=8e23c1ccb69c4d5292e05c66e5c4b81b].

Discussion

Consistent significant overall effects emerged regarding how participants were treated in the Cyberball game. Replicating a wealth of prior research (e.g., Ren et al., 2017), overincluded and included participants reported greater need satisfaction and more positive mood than excluded participants. Overincluded and included participants did not differ on need satisfaction or mood, consistent with some prior research (e.g., Carter-Sowell et al., 2010 Study 3; Kawamoto et al., 2012; Williams et al., 2000; Wolf et al., 2015).

Of particular interest and consistent with our primary hypothesis, we obtained some evidence that explanation condition moderates the effects of how participants were treated in the Cyberball game. In the control condition in which participants were only told that the purpose of the study was to investigate mental visualization, overincluded participants reported greater need satisfaction than both included and excluded participants. However, this pattern changed when participants were told that the purpose of the study was to investigate women’s mental visualization specifically. In this condition, the female participants, who were always playing with two men, had reason to believe their treatment was due to their gender. Here, overincluded women again reported greater need satisfaction than excluded participants but similar need satisfaction to included participants. This pattern of moderation was similar with regard to mood, although the difference between overinclusion and inclusion in the control condition did not reach significance, perhaps due to the less consistent mood effects observed in exclusion and inclusion work (Bernstein & Claypool, 2012).

Note although the results look largely similar despite the sample, hypothesized effects were stronger among the entire sample than the sample excluding participants who failed the explanation condition manipulation check. All non-significant omnibus and post-hoc tests in the complete sample analyses remain so when those who failed the explanation manipulation check are removed. All significant omnibus and post-hoc tests in the complete sample analyses remain so when those who failed the explanation manipulation check are removed with two exceptions. The omnibus interaction between treatment and exclusion conditions becomes non-significant on both need satisfaction and mood when those who failed the explanation manipulation check are removed. As noted above, the post-hoc patterns and associated significance remain the same. A likely explanation for the similar patterns of findings but differences in statistical significance is that the subtlety of the explanation condition manipulation and removal of participants who failed the associated manipulation check resulted in the limitation of reduced power. However, in general, these findings are consistent with the hypothesis that having reason to attribute one’s treatment to one’s gender eliminated the relative benefits of overinclusion as opposed to inclusion.

Although these findings are promising, we sought to test our hypothesis using another exclusion paradigm. Cyberball is a classic way in which to expose participants to actual differential treatment in the lab. However, this paradigm may have some drawbacks when it comes to studying our key moderator of explanation for treatment. Indeed, we sought to manipulate explanation subtly through the cover story that participants were given about the purpose of the Cyberball game. Although the majority of participants correctly answered our explanation manipulation check, the number of participants who missed this manipulation check suggests that our manipulation was in fact subtle. However, a more explicit manipulation of explanation in the Cyberball paradigm may have led participants to be suspicions regarding both the study research questions and the use of confederates in Cyberball game. Thus, we implemented a scenario method in a follow-up study which allowed for a more seamless explicit manipulation of explanation for treatment.

Study 2

Study 2 extended Study 1 by testing our key hypothesis using a different paradigm. Study 2 employed a scenario study in which participants imagined a scenario that fit a variety of criteria. Thus, this paradigm allowed for a more direct and explicit manipulation of explanation for treatment and therefore greater methodological control. The design of Study 2 was also simplified, focusing on only our key inclusion and overinclusion comparison and thus not including an exclusion condition.

Methods

Participants and design

Participants were female MTurk workers (n = 268, M_age = 38.26) who participated in exchange for $1. The majority of participants were Caucasian (n = 206), with others indicating African-American/Black (n = 23), Asian/Pacific Islander (n = 15), or other identifications (n = 24). Participants were randomly assigned to one condition of a 2 (Treatment Condition: Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) between-subjects design. Sample size was determined by ensuring at least 50 participants per cell, and then recruiting more than 200 participants in total in anticipation of needing to exclude some participants. This study was approved by the Amherst College Institutional Review Board (#16–010) and was not preregistered.

Procedure

Participants signed up for an online study entitled ‘Group Interactions’ which they were told investigated how people respond to different types of group interactions. After completing a consent form, participants were told that they were going to read about a group interaction and were given detailed instructions that encouraged them to imagine that they were actually experiencing the scenario in the present moment.

Participants were randomly assigned to imagine a scenario where they were either overincluded or included and this treatment was either due to their gender or their personality. All participants were asked to imagine that they were joining a new group that focused on an activity of interest to them and was comprised of mostly men. Participants were either asked to imagine that ‘you are being included in the group interaction and receiving the same attention that other group members are’ or that ‘you are being overincluded in the group interaction and receiving more attention than other group members are.’ Participants were also asked to imagine that ‘over the course of the meeting, you begin to suspect that you are being included/overincluded because of your personality/gender.’ See Appendix A for the complete scenarios.

Participants then completed measures of need satisfaction and mood (Williams, 2009). These measures were now asked only once, as participants were imagining a scenario. Need satisfaction was assessed with 12 items assessing the four fundamental needs of belonging (e.g., ‘I would feel disconnected’), self-esteem (e.g., ‘I would feel good about myself’), control (e.g., ‘I would feel powerful’) and meaningful existence (e.g., ‘I would feel meaningless’) answered on a scale from 1 (not at all) to 7 (extremely). These items were reverse-coded when necessary and combined to form a single measure of need satisfaction (α=.91). Mood was assessed with 9 items (e.g., ‘I would feel positive’) answered on a scale from 1 (not at all) to 7 (extremely). These items were reverse-coded when necessary and combined to form a single measure of mood with higher numbers indicating a more positive mood (α=.94).

To assess the explanation manipulation, participants completed the item ‘The way the group members interacted with me was due to’ and could choose between the options: my gender, my personality, or neither of the above. To assess the treatment manipulation, participants indicated on a 7-point scale how much attention they received during the group interaction, with 1 indicating not enough attention and 7 indicating too much attention. Finally, participants answered demographic questions and were debriefed.²

Results

Explanation manipulation check

Responses to the explanation manipulation check were significantly associated with explanation condition, X²(1, N = 268) = 147.50, p < .001. A 83% of participants in the gender explanation condition and 78% of participants in the control condition correctly indicated whether the way the group members interacted with them was due to their gender or their personality. Note that fewer participants failed this manipulation check than in Study 1, and those who failed the Study 2 manipulation check were roughly equally distributed across condition. The below analyses were conducted as intended with those 52 participants who failed the explanation manipulation check removed. Although a priori power analyses were not conducted, sensitivity analyses for main effects and interactions were conducted again using G*power (Faul et al., 2007). Given this final sample size, effects of Cohen’s f = 0.19 could be detected with 80% power.

Treatment manipulation check

The treatment manipulation check was analyzed with a 2 (Treatment Condition: Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) between-subjects ANOVA. A significant main effect of treatment condition emerged, F(1, 212) = 407.23, p < .001, η_p²=.65, whereby overincluded participants reported receiving more attention during the group interaction (M = 6.75, SD = 1.64) than included participants (M = 4.67, SD=.93). A much smaller significant main effect of explanation condition also emerged, F(1, 212) = 18.15, p < .001, η_p²=.08. Participants in the gender explanation condition reported receiving more attention (M = 5.95, SD = 1.22) than participants in the control condition (M = 5.51, SD = 1.37).

A treatment condition by explanation condition interaction also emerged on the treatment manipulation check, F(1, 212) = 5.02, p=.026, η_p²=.02. In the gender explanation condition, overincluded participants reported receiving more attention during the group interaction (M = 6.86, SD=.44) than included participants (M = 5.00, SD = 1.05), F(1, 212) = 163.94, p < .001, η_p²=.44. In the control condition, overincluded participants also reported receiving more attention during the group interaction (M = 6.65, SD=.78) than included participants (M = 4.33, SD=.65), F(1, 212) = 246.79, p < .001, η_p²=.54, although this effect was more pronounced.

Primary Analyses

Primary analyses consisted of 2 (Treatment Condition: Inclusion vs. Overinclusion) × 2 (Explanation Condition: Gender vs. Control) between-subjects ANOVAs on need satisfaction and mood.

Need Satisfaction

A significant main effect of explanation condition emerged, F(1, 212) = 50.65, p < .001, η_p²=.19, whereby participants in the control condition reported greater need satisfaction (M = 5.94, SD=.76) than participants in the gender explanation condition (M = 5.04, SD = 1.09). No significant main effect of treatment condition emerged, F(1, 212) = 1.04, p=.308, η_p²=.01.

Consistent with our hypothesis, a treatment condition by explanation condition interaction emerged, F(1, 212) = 4.64, p=.032, η_p²=.02 (see Figure 2). In the control condition, included participants reported greater need satisfaction than overincluded participants, F(1, 212) = 4.95, p=.027, η_p²=.02. However, in the gender explanation condition included and overincluded participants reported similar need satisfaction, F(1, 212) = 0.65, p=.420, η_p²=.00.

Figure 2. The effects of treatment and explanation conditions on need satisfaction in Study 2.

Error bars represent one standard error.

Mood

A significant main effect of explanation condition emerged, F(1, 212) = 42.61, p < .001, η_p²=.17, whereby participants in the control condition reported more positive mood (M = 6.01, SD = 1.04) than participants in the gender explanation condition (M = 4.96, SD = 1.36). A significant main effect of treatment condition also emerged, F(1, 212) = 8.99, p=.003, η_p²=.04, whereby included participants reported more positive mood (M = 5.72, SD = 1.24) than overincluded participants (M = 5.24, SD = 1.36).

Consistent with our hypothesis, a treatment condition by explanation condition interaction emerged, F(1, 212) = 5.08, p=.025, η_p²=.02 (see Figure 3). In the control condition, included participants reported more positive mood than overincluded participants, F(1, 212) = 13.55, p < .001, η_p²=.06. However, in the gender explanation condition included and overincluded participants reported similar mood, F(1, 212) = 0.28, p=.596, η_p²=.00.

Figure 3. The effects of treatment and explanation conditions on positive mood in Study 2.

Error bars represent one standard error.

Discussion

Consistent with both Study 1 and some prior research (e.g., Carter-Sowell et al., 2010 Study 3; Kawamoto et al., 2012; Williams et al., 2000; Wolf et al., 2015), the lack of a treatment main effect demonstrates that overincluded participants and included participants reported similar need satisfaction overall. However, included participants reported greater positive mood than overincluded participants. Although most of the prior research comparing overinclusion and inclusion has found no differences or a preference for overinclusion, this finding is consistent with a modest number of prior studies in the inclusion and exclusion literature (van Beest & Williams, 2006; van Beest et al., 2011) as well as more general studies on the drawbacks of feeling spotlighted (e.g., Crosby et al., 2014; Zou & Cheryan, 2015). Thus, this work provides some additional evidence that there are cases in which overinclusion is not experienced as positively as equity.

Of particular interest and again consistent with our primary hypothesis, explanation condition moderated the effects of how participants, all of whom were women, were treated in the scenario they imagined. In the control condition in which participants imagined their group experience was due to their personality, included participants reported greater need satisfaction and more positive mood than overincluded participants. However, when participants imagined that their group experience was due to their gender, included and overincluded participants reported similar need satisfaction and positive mood. Although Study 2’s pattern of findings is slightly different from the one that emerged in Study 1 given that it found that inclusion could be preferable to overinclusion, it again suggests that explanation for treatment is an important moderator that can contribute to the mixed findings on the relative benefits of inclusion vs. overinclusion. Consistent across both studies and with our hypothesis, differences between the experiences of inclusion and overinclusion were eliminated when participants were given reason to attribute their treatment solely to their gender identity. In addition, both studies underscore the fact that overinclusion experiences should not be assumed to be preferable to inclusion.

Notably, unlike in Study 1, the treatment manipulation check analyses in Study 2 yielded the predicted treatment condition effects but also unanticipated explanation condition effects. Participants in the gender explanation condition reported receiving more attention than those in the control condition, which may help explain why participants in Study 2 reported more need satisfaction and positive mood in the control explanation condition than the gender explanation condition. The treatment condition wording was identical regardless of whether participants were imagining the control or gender explanation. Thus, this unintended spillover effect of the explanation manipulation is likely an indication that when participants were asked to imagine a situation where their treatment was due to their gender they tended to naturally associate this with more attention than situations where their treatment was not due to their gender. Importantly, the fact that treatment due to one’s gender may be naturally associated with more attention suggests overinclusion due to one’s gender may be particularly common in everyday experiences. However, the unanticipated relationship between explanation condition and perceptions of treatment undermines the ability to make clear, independent conclusions about the effects of these variables separately. Thus, future research would benefit from using paradigms like Cyberball that more objectively manipulate treatment in explicitly quantifiable ways, thus reducing the likelihood that perceptual biases confound explanations with treatment.

General discussion

The primary purpose of the current work was twofold: to explore circumstances under which experiences of inclusion and overinclusion may not be so positive and to investigate a potential novel moderator of inclusion and overinclusion experiences: the explanation for this treatment. As previously noted, past research regarding the experience of overinclusion vs. inclusion is quite mixed. In general, the current work suggests that overall, the experience of overinclusion and inclusion is relatively similar. Overall, in Study 1 participants who were included and overincluded experienced similar need satisfaction and positive mood and in Study 2 participants who were included and overincluded experienced similar need satisfaction. These findings are consistent with a number of past studies (e.g., Carter-Sowell et al., 2010 Study 3; Kawamoto et al., 2012; Williams et al., 2000; Wolf et al., 2015). However, we did obtain some evidence that overinclusion is not always similar or preferable to inclusion. In Study 2 participants, all of whom were women imagining an interaction with men, who were included experienced more positive mood overall than participants who were overincluded. Thus, Study 2 demonstrates that when participants are asked to imagine being overincluded in a group conversation where they are in the gender minority, they feel less positive mood than imagining being treated similarly to other group members and thus simply included. Perhaps, this difference is a function of the different paradigms. Overinclusion may not feel very awkward when it involves receiving a virtual ball in Cyberball but may feel more uncomfortable when it involves imagining a real-life situation which includes multiple indicators of overinclusion including receiving more eye contact, more questions, more opportunities to speak, and more responses directed at you. This explanation is in line with the fact that much of the evidence that overinclusion can be a negative experience uses paradigms with higher external validity than Cyberball (Crosby et al., 2014; Settles et al., 2019). Thus, this work contributes to a small but growing literature that challenges the assumption that inclusion and overinclusion are necessarily positive experiences and primarily useful to study as comparison conditions for exclusion (McCarty et al., 2022).

Importantly, the current work also demonstrated that the explanations we make for our treatment can be an important moderator of our inclusion and overinclusion experiences. Indeed, explanation moderated the impact of treatment in both Studies 1 and 2. However, the interactive patterns differed across these studies. In Study 1, participants felt greater need satisfaction when they were overincluded than when included, provided they were given little reason to think their gender had something to do with their treatment. However, when given a reason to think that their gender had something to do with their treatment, the differences between overinclusion and inclusion were eliminated. Although similar moderation patterns were obtained on mood, the difference between overinclusion and inclusion did not reach statistical significance in the control condition. These post-hoc patterns were significant regardless of sample, but notably the number of participants who failed the explanation manipulation check suggests that the manipulation was subtle, and the omnibus interactive effects were stronger and statistically significant with the increased power of the entire sample. In Study 2 which used a more explicit explanation manipulation, participants felt greater need satisfaction and positive mood when they were included than when they were overincluded, provided they were given little reason to think their gender had something to do with their treatment. However, when given a reason to think that their gender was the primary reason for their treatment, the differences between overinclusion and inclusion were eliminated.

There are a number of potential reasons for these differential patterns of moderation. One potential explanation is the different paradigms utilized by Studies 1 and 2. Although past research using Cyberball has itself been mixed as to whether inclusion or overinclusion is preferable, previous Cyberball studies are often consistent with our Study 1 findings that overinclusion can lead to greater need satisfaction and mood than inclusion (e.g., Kwok et al., 2018; Niedeggen et al., 2014; van Beest et al., 2011). Participants may experience actual inclusion and overinclusion in Cyberball differently than they experience imagined inclusion and overinclusion scenarios. For example, people may mispredict how they react in inclusion situations (McCarty et al., 2018; McCarty et al., 2021), predicting they may find overinclusion uncomfortable (Study 2) when actual disproportionate attention may actually feel validating (Study 1). Alternatively, participants may find that overinclusion in the intentionally sterile world of Cyberball is meaningfully different from more multifaceted or complex situations like those imagined in Study 2. Thus, perhaps overinclusion may be positive in controlled, low stakes environments like Cyberball, but more negative in the types of more nuanced situations participants imagined from real life. Another potential explanation for these different patterns of findings is that the moderating effects of explanations for treatment depend on how subtle these explanations are. Study 1 utilized a more subtle, indirect manipulation of explanation, as participants were told that the researchers’ focus (not the other participants’ focus) was either on understanding mental visualization in general or women’s mental visualization specifically. This choice was made to reduce suspicion but may have created attributional ambiguity that led to more neutral or positive attributions and thus more positive overinclusion experiences. In contrast, Study 2’s scenario paradigm allowed for a direct, relatively unambiguous manipulation of explanation, as participants were either instructed to imagine their treatment was due to their gender or their personality. Overinclusion may be experienced more negatively when the attribution for this treatment is clearly one’s gender. Thus, future research may explore the role of attributional ambiguity in these effects. Despite the different patterns of effects observed, it is notable that explanation was a significant moderator of overinclusion and inclusion experiences in both Studies 1 and 2 and that having good reason to attribute treatment to one’s gender consistently eliminated differences between inclusion and overinclusion in both Studies 1 and 2.

Although the different paradigms may have contributed to our differential patterns of findings, a strength of our current work is that our hypothesis was tested in two different paradigms with complementary strengths and weaknesses. Although Study 1 put participants in an actual inclusion or overinclusion situation via Cyberball, this ball tossing situation is not likely one participants regularly experience in everyday life. Additionally, Study 1 employed a subtle manipulation of the explanation for treatment. On the other hand, Study 2 asked participants to imagine an inclusion or overinclusion situation. Although they did not experience an actual inclusion or overinclusion situation, the imagined scenario likely allowed participants to think about a more relatable everyday life experience than Cyberball. In addition, the scenario instructions allowed for a more explicit and clear manipulation of explanation of treatment. The current work also used both student (Study 1) and broader samples (Study 2), demonstrating the importance of explanations for inclusion and overinclusion across people with varying backgrounds and life experiences. Future research should consider whether there are ethical paradigms in which participants could actually experience overinclusion due to their gender in a lab setting that is high in external validity, perhaps using trained confederates to enact situations similar to the scenarios utilized in Study 2.

In this first investigation into how explanations for treatment may moderate the effects of inclusion and overinclusion, we focused on gender as a potential explanation. We did so given the modest literature on the negative effects of inclusion and overinclusion generally focused on aversive experiences regarding treatment due to one’s social identities (Crosby et al., 2014; Settles et al., 2019). Future research may benefit from exploring other potential moderators of inclusion and overinclusion experiences that may help explain the currently inconsistent literature. Although this work focuses on situational moderators, intrapersonal factors like ability and motivation to process the inclusion or overinclusion experience may affect the degree to which differences between inclusion and overinclusion are observed. Other situational factors may also be fruitful moderators. For example, past work has explored the role of granted vs. claimed inclusion and overinclusion (de Waal-Andrews & van Beest, 2012; De Waal-Andrews & van Beest, 2020). However, one common example of overinclusion is experiencing more attention and recognition than other group members because it is earned by virtue of an award, suggesting superior performance. Inclusion and overinclusion, where it is earned due to relatively objective high performance, may be more likely to be experienced positively and less likely to be awkward or uncomfortable. Thus, future work may compare granted vs. claimed vs. earned inclusion and overinclusion.

In both studies, we chose to investigate women’s experiences in groups of men. Thus, a limitation of the current work is that our findings are only generalizable to women’s experiences in groups of men. However, people with other gender identities, such as transgender and gender non-conforming people, may be especially likely to be unrepresented in groups and wonder whether their treatment is due to their gender identities. Thus, future research may benefit from exploring the experiences of gender diverse people. Although we would expect to find similar patterns on need satisfaction and mood, more specific questions regarding participants’ cognitions during the interaction may yield interesting differences. Women and gender non-conforming people may be similarly likely to attribute this differential treatment to prejudice, whereas gender non-conforming people may be more likely than women to consider ignorance or discomfort as a potential explanation and women may be more likely to consider potential romantic or sexual interest as a potential explanation. Men’s experiences attributing their inclusion and overinclusion to gender may also be interesting to explore, as their general experiences of privilege and power may attenuate the potentially negative effects of attributing treatment to their identity. The role of explanations regarding social identities outside of gender, including race, are also important to study. Given the current political climate in the United States including the Black Lives Matter movement and legislation regarding critical race theory, people of color may be especially aware of whether they are underrepresented in groups and vigilant regarding differential treatment, perhaps making the effects observed in the current study even more pronounced when it comes to race as opposed to gender.

The identities of the other group members are also likely to impact the experience of inclusion and overinclusion as well as attributions to bias (Goodwin et al., 2010). Notably, although the identities beyond the gender of the other group members were unspecified in Study 2, names were chosen for the other group members in Study 1. The names, Matt and Brian, may bring with them assumptions of race, age, personality, sexuality etc. (Dahl & Krog, 2018) that may moderate the observed effects. Future research may benefit from exploring these other aspects of the group members as moderators or from measuring these aspects and statistically controlling for them in analyses. For example, a future study could manipulate the sexuality of the male group members and thus determine whether the observed effects of women’s overinclusion are a function of assumptions that the men they are interacting with may be expressing romantic or sexual interest. Notably, assumptions of sexual attraction may have contributed to participants’ experiences in Study 2, but they are less likely to have had a strong role in Study 1 where participants were interacting online and only had access to each other’s names.

The current work investigated overinclusion using the most common outcome variables in the social exclusion literature, need satisfaction and mood. Although a strength of this choice is that it allows for comparisons to many other overinclusion studies, particularly those that compare overinclusion and inclusion, the concept of overinclusion relates to many other literatures including research on tokenness, affirmative action, and the minority spotlight effect (Crosby et al., 2014; Turner & Pratkanis, 1994; Zou & Cheryan, 2015). As such, future research would benefit from exploring outcome variables and methods outside of those used in the social exclusion literatures. For example, self-determination theory (Ryan & Deci, 2017) posits autonomy, competence, and relatedness as basic human needs. Although there are conceptual overlaps between these needs and the need satisfaction measures of belonging, control, self-esteem, and meaningful existence, future work could assess autonomy, competence, and relatedness directly.

Although a number of prior studies have compared the experience of inclusion and overinclusion, these studies have not generally focused on this comparison. The current work begins to fill the gap in this literature by explicitly focusing on the comparisons between inclusion and overinclusion, highlighting the mixed previous literature, and suggesting that future work focus on moderators. We find initial evidence that the explanation for the treatment affects the experience of inclusion and overinclusion. Across two studies using two different paradigms, differences between inclusion and overinclusion were eliminated when participants had reason to think their treatment was due to their gender and inclusion and overinclusion experiences were not always similarly positive. The current work suggests that in an effort to have more diverse groups, including or even overincluding members of traditionally underrepresented groups is not sufficient for members of these communities to have positive group experiences; the explanations they have for their treatment matters. Thus, the current work echoes the cries of diversity, equity, and inclusion experts that recruiting and creating diverse groups is only the first step toward productive intergroup relationships. More work needs to be done to ensure positive experiences and working relationships once these groups are created. We hope this work inspires more research investigating inclusion and overinclusion, as well as research on how these experiences can be affected by our social identities.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The Study 2 dataset is available upon request from the first author. The consent form participants signed in Study 1 indicated that their data would not be shared outside of the immediate research team, and thus prevents us from sharing the Study 1 dataset publicly.

Notes

1. Although not of primary interest to the current work, need satisfaction and positive mood were assessed both reflexively and reflectively, consistent with much of the exclusion literature and the temporal need-threat model (Williams, 2009). This model posits that we detect exclusion rapidly and crudely given that in our evolutionary history, expulsion from groups likely meant death. However, these reflexive reactions dissipate quickly (Wesselmann et al., 2012) once people are able to reflect on the situation and make attributions based on situational context. Analyses of the timing effects in Study 1 yielded findings consistent with this rationale. Complete reports of the timing effects can be found here: [https://osf.io/ebq2k/?view_only=8e23c1ccb69c4d5292e05c66e5c4b81b].

2. In both studies, additional demographic information not mentioned in text and personality and exploratory items were also included to aid future research. These variables include attachment style, strength of gender identification, system justification, need to belong, need for uniqueness, perceived group cohesion, perceived group warmth, perceived group competence, expectations regarding treatment, in-group comparisons, a description of what participants imagined during the mental visualization task in Study 1, participants’ memories for names of the other players in the mental visualization task in Study 1, and the number of group members imagined in Study 2. All of the measures are available from the first author.

Appendix A

Included Scenarios

Please imagine you are interested in joining a local group composed of mostly men. This group focuses on an activity of interest to you (choose a real personal interest – perhaps something like an extracurricular activity or charity work). You go to the group meeting for the first time. The group members include you in conversation, respond when you speak, and make eye contact with you. In other words, you are being included in the group interaction, and receiving the same attention that other group members are. This behavior continues throughout the group meeting. As the group works, the other group members are looking in your direction, asking you questions, and responding to your statements. In general, the group members are being as responsive to you as they are to the other group members. Over the course of the meeting, you begin to suspect that you are being included because of your personality/gender.

Overincluded scenarios

Please imagine you are interested in joining a local group composed of mostly men. This group focuses on an activity of interest to you (choose a real personal interest – perhaps something like an extracurricular activity or charity work). You go to the group meeting for the first time. The group members go out of their way to include you in conversation, always respond when you speak, and constantly make eye contact with you. In other words, you are being overincluded in the group interaction, and receiving more attention than other group members are. This behavior continues throughout the group meeting. As the group works, the other group members are constantly looking in your direction, asking you questions, and responding to your statements. In general, the group members are being more responsive to you than they are to the other group members. Over the course of the meeting, you begin to suspect that you are being overincluded because of your personality/gender.