The look in your eyes: The role of pupil dilation in disguising the perception of trustworthiness

Abstract

Pupil size reflects the cognitive and affective states of the beholder and thus shapes interpersonal impressions. Individuals with dilated pupils are evaluated more positively than those with constricted pupils. The present study investigated the role of pupil dilation in building interpersonal trust. We used face photographs taken by Okubo et al. (2017), in which models (N = 81) were posed as trustworthy persons for a photograph shown in an economic game. We measured the pupil diameter of each model’s photographs using image processing software. The pupils were dilated when the models expressed trustworthiness in their faces. Moreover, untrustworthy choices in an economic game predicted pupil dilation. As dilated pupils produce positive impressions, the results suggest that pupil dilation may be associated with concealing signals of trustworthiness. Untrustworthy individuals may use pupillary responses almost incapable of voluntary control to exploit others in social interactions.

KEYWORDS:

• Facial trustworthiness
• economic game
• pupil dilation

Introduction

Trust plays a vital role in the development and durability of cooperation (Ross & LaCroix, 1996; Simpson, 2007). As there is more uncertainty than certainty, trusting others can be a strategy for reducing complexity and risk in cooperative social interactions (Simpson, 2007). Thus, discriminating between trustworthy and untrustworthy is an essential social skill (Todorov & Oosterhof, 2011). Due to this essentiality, several theories advocate that humans have evolved innate mechanisms specialised in processing facial features to detect trustworthiness (e.g. Van’t Wout & Sanfey, 2008; Wood, 2020). Supporting these theories, researchers have discovered that infants are sensitive to facial signs of trustworthiness, as shown by behavioural (Sakuta et al., 2018) and neuroscientific evidence (Baccolo et al., 2021; Jessen & Grossmann, 2016). Adults can use this innate mechanism to discriminate between trustworthy and untrustworthy facial expressions. For example, individuals with trustworthy faces benefit from economic games (e.g. Chang et al., 2010; Stirrat & Perrett, 2010), online commercial activities (Barnes, 2021; Duarte et al., 2012; Ert et al., 2016), and criminal sentencing (Wilson & Rule, 2015). Judgments about trustworthiness can be based on various facial cues (for a review, see Foo et al., 2022; Todorov, 2017). Some of these cues are dynamic, such as emotional expressions (Krumhuber et al., 2007), whereas others are stable, such as face width (Stirrat & Perrett, 2010), gender (Buchan et al., 2008), and age (Li et al., 2022). Some researchers have suggested that dynamic cues are more helpful for trustworthiness judgments (Hehman et al., 2015), while others have suggested that stable cues are more important because they have biological and evolutional bases (Rezlescu et al., 2012). Emotional expressions can work in a dynamic (i.e. movies, Krumhuber et al., 2007) as well as in a stable format (i.e. still images, e.g. Li et al., 2022; Todorov et al., 2008; Todorov & Oosterhof, 2011) although the former is more effective than the latter as trustworthiness cues (Krumhuber et al., 2007). Wood (2020) mathematically formulated the processes of building interpersonal trust using innate mechanisms for facial information and extended it to modern marketing exchanges and management.

Although several theories posit innate mechanisms to detect facial trustworthiness (e.g. Van’t Wout & Sanfey, 2008; Wood, 2020) and received some empirical support (e.g. Baccolo et al., 2021; Jessen & Grossmann, 2016; Sakuta et al., 2018), trustworthiness detection is typically inaccurate. Participants discriminated between trustworthy and untrustworthy partners in prisoners’ dilemma games, but their accuracy was approximately 60% (Frank et al., 1993; Verplaetse et al., 2007). Foo et al. (2022) conducted meta-analyses across 25 studies with 1,976 faces and 3,500 participants. They found that people have difficulty discriminating trustworthy faces from untrustworthy ones; the correlation between perceived and actual trustworthiness was, at most, modest (r = .14). Although Ekman and his colleagues argued that micro-expressions, which are very brief, involuntary facial expressions, can reveal whether a person is being deceitful (Ekman, 2003; Ekman & O'Sullivan, 1991; O’Sullivan & Ekman, 2004), this argument has been criticised both theoretically (e.g. Burgoon, 2018) and empirically (e.g. Bond Jr & Uysal, 2007; Porter & ten Brinke, 2008). According to Hamilton (1964a,b), evolutionary pressure exists not only for cooperation by increasing inclusive fitness (or deception detection, Cosmides, 1989) but also for deceiving others by concealing untrustworthy signals. Frank (1988) noted that detecting and concealing untrustworthy cues may resemble a predator-prey arms race, where frequency-dependent oscillations between signal detection and signal deception occur over evolutionary time.

Based on the predator-prey idea (Frank, 1988), we propose the signal concealment hypothesis to explain the difficulty of trustworthiness judgements on faces (Okubo et al., 2012, 2017). The signal concealment hypothesis focuses on the role of fake smiles in building trust. A smile is the easiest expression to fake and is often used to mask true feelings (Ekman & Friesen, 1982). Smiling partners, relative to partners with neutral expressions, earn more monetary rewards in economic games (Centorrino et al., 2015; Krumhuber et al., 2007). Importantly, even a posed smile increases perceived trustworthiness (Krumhuber et al., 2007). Todorov and his colleagues developed a computer model and demonstrated the relationship between perceived positive expressions (i.e. smiles) and trustworthiness (Todorov et al., 2008; Todorov & Oosterhof, 2011; Oosterhof & Todorov, 2008). Based on the relationship between perceptions of positive emotions and trustworthiness, the signal concealment hypothesis proposes that a fake smile is used to conceal signals of untrustworthy attitudes or traits, leading to difficulty in judgements based on facial trustworthiness (Okubo et al., 2012). We reasoned that smiles not only mask one’s true feelings (Ekman & Friesen, 1982) but also disguise signals of trustworthiness derived from stable facial features (e.g. face width, Stirrat & Perrett, 2010), which are assumed to be unfakeable (Rezlescu et al., 2012). Supporting our hypothesis, successful judgments of facial trustworthiness were thwarted when the models posed a fake smile (Okubo et al., 2012, 2017, 2018; Okubo & Ishikawa, 2019). In addition, people tend to adopt effective strategies to conceal signals of untrustworthiness (Okubo et al., 2017). Emotional expressions are expressed more intensely in the left hemiface than in the right (e.g. Indersmitten & Gur, 2003; Sackeim et al., 1978; Zaidel et al., 1995). Okubo et al. (2017) found that participants tended to show the emotional side of the face (i.e. the left side) when asked to express trustworthiness for a portrait shown in an economic game. This tendency was larger for untrustworthy individuals who were deceived greatly in the economic game than for trustworthy individuals. In addition, untrustworthy individuals showing the left side of the face were rated as emotionally more positive (Okubo & Ishikawa, 2019) and just as trustworthy individuals (Okubo et al., 2017).

The purpose of the present study was to conduct a further test of the signal concealment hypothesis, focusing on pupil responses, which occur spontaneously and are thus difficult to control voluntarily (Loewenfeld, 1993). Pupils dilate or constrict not only in response to changes in ambient light but also to changes in mental state, such as an increase in cognitive demand (Kahneman, 1973; Just et al., 2003) or emotional arousal (Bradley et al., 2008; Hess, 1965). Pupil responses to mental states are controlled by a subcortical mechanism of the parasympathetic oculomotor complex, the noradrenergic system in the locus coeruleus, which autonomously regulates arousal and the sleep-wake cycle (Wilhelm et al., 1999). Since pupil responses are difficult to control voluntarily, they can be taken as a genuine measure of a mental state (see Laeng et al., 2012 for a review) and thus have a substantial effect on evaluations from others (Hess, 1975). Hess’s seminal papers (1965, 1975) provide an excellent example: faces with dilated pupils are perceived as more attractive than those with constricted pupils. The observer may associate pupil dilation with the models’ positive impression (coupled with heightened arousal) of the observer, resulting in a higher attractiveness rating.

Pupil dilation is associated not only with facial attractiveness but also with facial trustworthiness. Kret and De Dreu (2019) presented face photographs (seven degrees of visual angle) of dilated and constricted pupils in a trust game. The size of the pupil was dilated by 140% or constricted by 60% of the original diameter. Kret and De Dreu demonstrated that participants trusted game partners with dilated pupils more than those with constricted pupils. In addition, pupil dilation can act as a dynamic cue of trustworthiness, which is more effective than stable cues (Hehman et al., 2015; Krumhuber et al., 2007). Taking into account the association between pupil dilation and facial trustworthiness, we hypothesised, based on the signal concealment hypothesis, that untrustworthy individuals conceal their attitudes or traits through pupil dilation. To test this hypothesis, we used face photographs taken by Okubo et al. (2017), where the models were considered as trustworthy as possible to allow the game partner’s trust. In addition, we used facial photographs with neutral expressions as a baseline for measuring pupil dilation. Our predictions are as follows: (1) the number of untrustworthy choices (i.e. the number of cancellations as the second mover) in the trust game predicts pupil dilation, and (2) the relationship between untrustworthy choices and pupil dilation would be more evident in models showing the left cheek than in those showing the right cheek.

Materials and methods

Stimulus materials

The facial photographs taken in Okubo et al.’s (2017) study were used in the present study. A digital camera (Canon EOS M) with an interchangeable lens (Canon EF-S 55–250 mm f/4.0-5.6 IS STM) was used to capture the images. Eighty-one right-handed participants (M_age = 19.49, SD = 1.35) were recruited from Senshu University, Kawasaki, Japan. Handedness was assessed using the Japanese version of the FLANDERS handedness questionnaire (M = 9.41, SD = 1.08; Nicholls et al., 2013; Okubo et al., 2014). As we used photographs taken in a previous study (Okubo et al., 2017), an a priori power analysis was not conducted to determine the sample size. However, a sample size of 81 yielded a power of .80, assuming a medium effect size of r = .30 and α = .05. This study was approved by the Senshu University Human Research Ethics Committee, and the participants provided written informed consent before participating in the study.

In the photoshoot, the models were asked to pose trustworthy people to help their game partners develop their trust. When the models looked straight at the camera, they were asked to pose again by turning their faces slightly to the left or the right. Photographs of neutral expressions were also taken during the photo session.

After the photoshoot, the models performed a computer version of the trust game to assess the level of (un)trustworthiness. In the trust game, each model was encouraged to earn as much money as possible through trade with a virtual counterpart whose facial photograph was presented on a computer screen. Ninety-six facial photographs were selected from the Asian Image Database PF01 (2001) as the images of their counterparts. All the models were Korean university students of the same age as the participants. The game was designed as a one-shot game; therefore, the image of each counterpart appeared only once during the experiment. Participants were led to believe that they were competing against an opponent at a separate computer terminal when, in fact, no opponent existed. The choices made by their virtual opponents were pre-preprogrammed: they traded and cancelled as first or second movers randomly and evenly (i.e. 50% of trials). Okubo et al. (2017) chose the game with virtual opponents because (1) the number of untrustworthy choices (see below) was measured using the same procedure for each model. In addition, (2) the game with virtual opponents could save time and effort, and (3) it raised fewer ethical issues than the game with human opponents.

In each trial, the model was randomly assigned to either the first or second mover. The monetary reward for the trade was dependent on the choices made by the model as follows: (1) if both the first and second movers chose to conduct the trade, both received 650 yen (US$ 6.50); (2) if the first mover cancelled the trade, both movers received JPY 500 (US$ 5); and (3) if the first mover conducted the trade but the second mover cancelled it, the first mover received only 300 yen (US$ 3), while the second mover received 1,000 yen (US$ 10). After 96 trials, the model received a monetary reward at an exchange rate of 100 yen for every 1,000 yen earned during the game. The number of trades as the first mover was an operational measure of trust, whereas the number of cancellations as the second mover was a measure of untrustworthiness (Stirrat & Perrett, 2010). Thus, we used the latter score as the untrustworthiness score (M = 16.46, SD = 6.99).

Pupil diameter measurement

We measured the pupil diameter and iris width of face photographs with trustworthy and neutral expressions. The measure tool of GIMP 2.10.30 (the GIMP Development Team), an image manipulation software, was used to obtain the pixel distance of the pupil diameter and iris width. If the contrast of the photograph was low and the pupil was difficult to recognise, the brightness was increased; if this was not sufficient, the contrast was enhanced.

Although the photographs of trustworthy and neutral expressions were taken from the same model, the iris widths of the two photographs did not necessarily match in size. This is due to slight differences in shooting conditions (e.g. head turn or tilt). Therefore, pupillary change was calculated by comparing trustworthy faces with neutral faces using pupil diameter ratio to iris width. Pupillary change is defined by Equation (1)(1) $PC={\displaystyle \frac{TP/TI}{NP/NI}}$(1) as follows: (1) $PC={\displaystyle \frac{TP/TI}{NP/NI}}$(1) where PC is the pupillary change, TP is the pupil diameter of the trustworthy faces, TI is the iris width of the trustworthy faces, NP is the pupil diameter of the neutral faces, and NI is the iris width of the neutral faces. A value of 1 indicated no change, greater than 1 indicated pupil dilation, and less than 1 indicated pupil constriction.

Results

Photographs of 13 models were excluded from statistical analysis because pupil diameter could not be measured due to image artefacts (e.g. intense highlight, closure of the eyes, pupil covered with eyelids). Assuming a medium effect size of r = .30 and α = .05, the sample size of 68 yielded a power of .73. We conducted a statistical analysis using photographs of the remaining 68 models. The mean of pupillary change was 1.04 (SE = 0.02). It was significantly larger than the value 1, indicating no change (t (67) = 2.52, p = .01, d = .31). In other words, pupil dilation was observed when the models expressed trustworthiness in their faces.

Figure 1 shows an increase in pupil dilation with the untrustworthiness score (i.e. the number of cancellations as the second mover). To examine this relationship, we conducted a multiple regression analysis on pupillary change with untrustworthiness, positing direction (left vs. right), and the interaction between these two variables as predictors. The posing direction was coded as 0 for the left cheek pose and 1 for the right cheek pose. To reduce multicollinearity in the regression, untrustworthiness was mean-centred before calculating the interaction term (Aiken & West, 1991). Table 1 presents the results of the multiple regression analysis. As shown in Figure 1, untrustworthiness significantly predicted pupillary changes (b = 0.008, β = .25, p = .009). In contrast, posing direction (b = 0.02, β = .08, p = .47) and the interaction term (b = −0.008, β = -.20, p = .10) were not significant.

Figure 1. The relationship between untrustworthiness scores and pupillary change. The untrustworthiness scores were the number of cancellations as the second mover in the trust game.

Table 1. The results of multiple regression analysis on pupillary change.

Predictors	b	β	p value	***
Uncooperative choices	0.008	0.25	0.01	**
Posing direction	0.024	0.09	0.47
Interaction	-0.008	-0.20	0.10

Discussion

In the present study, we tested our hypothesis that untrustworthy individuals would use pupil dilation to conceal their uncooperative attitudes or traits. Overall, the pupils dilated when the models expressed trustworthiness in their faces. In support of our predictions, the untrustworthiness score predicted pupil dilation when the model expressed trustworthiness. In contrast to our predictions, posing direction did not substantially affect the relationship between untrustworthiness and pupil dilation. Although our predictions are not fully confirmed, the relationship between untrustworthiness and pupil dilation fits well with the signal concealment hypothesis (Okubo et al., 2012, 2017, 2018; Okubo & Ishikawa, 2019). People trust those with dilated pupils more than those with constricted pupils (Kret & De Dreu, 2019). Our results suggest that untrustworthy individuals may use this relationship to conceal their attitudes or traits.

As pupil responses occur spontaneously and are difficult to control voluntarily (Loewenfeld, 1993), the models in the present study may not have voluntarily controlled their pupil responses when expressing trustworthiness. The enhancement of pupil dilation with an untrustworthiness score may be an epiphenomenal reaction to expressing trustworthiness. Loewenfeld (1993) listed possible indirect strategies to voluntarily change pupil size (e.g. self-induction arousal, imagination, Loewenfeld, 1993, p. 650, Table 1). An increase in cognitive demand and emotional arousal could be possible mechanisms to explain the enhancement of pupil dilation. Hess and Kleck (1990) pointed out that sending a deceptive signal is cognitively more demanding than sending an honest signal because the former is more complex and thus requires more cognitive effort than the latter. Furthermore, untrustworthy individuals, relative to trustworthy ones, should have been emotionally more aroused because they had a higher motivation to earn rewards; in particular, they received higher monetary rewards than trustworthy individuals (Okubo et al., 2017). Both these mechanisms may contribute to the enhancement of pupil dilation observed among untrustworthy individuals in the present study.

In contrast to our predictions, the interaction between posing direction and deception was not significant. The sample in the present study contained 43 models showing the left cheek and 25 models showing the right cheek. This uneven distribution of the posing direction may be responsible for the null interaction in the regression analysis, where the interaction term was calculated as the product of posing direction and untrustworthiness. Future research may employ a sample with a balanced distribution in the posing direction.

Although the untrustworthiness score predicted pupillary change, its effect size was modest in the present study (β = .25). Pupils change much more in response to light than to the psychological state; pupil size often doubles in response to light, while it increases by up to 20% according to the psychological state (Laeng et al., 2012). Illumination during the photoshoot may not have been sufficiently controlled, producing noise that interferes with the effect of deception on pupillary changes. However, it should be noted that the facial photographs were taken during the same experimental session with the same equipment and lighting. The pupillary change was defined as the ratio relative to the iris width, which was slightly affected by illumination. Furthermore, there were no systematic differences in illumination and shooting conditions (e.g. shooting distance) when taking photographs with trustworthy and neutral expressions. Thus, the pupillary change in the present study should reflect the psychological state, although it may contain some noise from illumination.

There are several limitations in the present study. First, we used existing data (face photographs taken by Okubo et al., 2017). Facial photographs were not collected to address the particular research question or to test the hypotheses of the present study. Moreover, it was impossible to draw a definitive conclusion from the present data because we did not manipulate the dependent variables. Second, we did not use genuine emotional expressions in the present study. Okubo et al. (2017) asked the models to pose as trustworthy as possible when they took face photographs. Ekman and his colleagues pointed out that micro expressions, very brief, involuntary facial expressions, can reveal whether a person is being deceitful (Ekman, 2003; Ekman & O'Sullivan, 1991; O’Sullivan & Ekman, 2004). Posed expressions used in the present study did not have such emotional information. Third, a causal relationship between the untrustworthiness score and pupil dilation was unclear, although the former predicted the latter. Direct manipulation of trustworthiness should be needed to draw an unequivocal conclusion on this relationship. Considering these limitations, careful interpretation and future research should surely be needed.

Conclusion

The present study demonstrated that the number of uncooperative choices in the trust game predicted pupil dilation when the models expressed trustworthiness in their faces, this supported the signal concealment hypothesis (Okubo et al., 2012, 2017, 2018, 2019), which explains the difficulty in detecting facial trustworthiness. Although facial trustworthiness can be processed automatically using the innate mechanisms (Baccolo et al., 2021; Jessen & Grossmann, 2016; Sakuta et al., 2018), trustworthiness detection is typically inaccurate because untrustworthy individuals conceal signals of untrustworthiness by use of a fake smile with dilated pupils. Kret and De Dreu (2019) showed that people trusted those with dilated pupils more than those with constricted ones. It is tempting to speculate that untrustworthy individuals may use this relationship to conceal their attitudes or traits through pupil dilation, which is difficult to control voluntarily (Loewenfeld, 1993). Pupil dilation larger in the untrustworthy than in the trustworthy may thwart the trustworthiness detection and play a role in a predator-prey arms race detecting and concealing untrustworthy cues (Frank, 1988), decreasing the accuracy of trustworthiness detection (for a review, Foo et al., 2022).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are openly available in zenodo at https://doi.org/10.5281/zenodo.6418414

Additional information

Funding

This work was supported by the Japan Society for the Promotion of Science KAKENHI under Grant 19K03384 to the first author.

Notes on contributors

Matia Okubo

Matia Okubo is a professor of cognitive psychology at Senshu University, Kanagawa, Japan. His work focuses on laterality and everyday cognition. Specific interests include handedness, footedness, facial expressions, trustworthiness, attention, perception, and locomotion – to name a few. He also conducts research on martial arts, which have fascinated him since his childhood.

Kenta Ishikawa

Kenta Ishikawa is an assistant professor of cognitive and clinical psychology at Senshu University, Kanagawa, Japan. His research interest focuses on attentional processing in anxiety disorders, with particular emphasis on the relationship between attention and maintenance of social anxiety.

Takato Oyama

Takato Oyama is a PhD student in cognitive psychology at Senshu University, Kanagawa, Japan. His research interests focus on human nonverbal communication, such as face cognition, gaze cognition, and attention.

Yoshihiko Tanaka

Yoshihiko Tanaka is a master’s student in cognitive psychology at Senshu University, Kanagawa, Japan. His research interests focus on social cognition, attention, and experimental psychology.