“Adding an egg” in algorithmic decision making: improving stakeholder and user perceptions, and predictive validity by enhancing autonomy

ABSTRACT

Decision makers often combine multiple pieces of information to make performance predictions and hiring decisions. More valid predictions are made when information is combined algorithmically (mechanical prediction) rather than in the decision-maker’s mind (holistic prediction). Yet, decision makers rarely use algorithms in practice. One reason is that decision makers are worried about negative evaluations from other stakeholders such as colleagues when using algorithms. We hypothesized that such stakeholders evaluate decision makers more positively when they use autonomy-enhancing algorithmic procedures (AEAPs, holistically adjust predictions from a prescribed algorithm or self-design an algorithm), than when they use a prescribed algorithm. Relatedly, we hypothesized that decision makers who use AEAPs are less worried about negative stakeholder evaluations, and more likely to use algorithms in performance predictions. In Study 1 (N = 582), stakeholders evaluated decision makers more positively when they used AEAPs rather than a prescribed algorithm. In Study 2 (N = 269), decision makers were less worried about negative stakeholder evaluations and more likely to use AEAPs compared to a prescribed algorithm. Importantly, using AEAPs also resulted in substantially higher predictive validity than holistic prediction. We recommend the use of self-designed algorithms to improve perceptions and validity.

KEYWORDS:

• Algorithm use
• personnel selection
• decision making
• human-automation collaboration
• decision-support

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary Material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/1359432X.2023.2260540

Notes

1. We note that decision makers could also be stakeholders themselves, such as when observing other decision makers making hiring decisions.

2. Stability, which is another key dimension, is the extent to which an outcome is perceived as stable or consistent over time and across trials. We had no theoretical argument how it would relate to stakeholder perceptions and only measured it for theoretical completeness.

3. The word “positive” reflects that stakeholders perceive decision-makers’ locus of causality to be higher (i.e., more internal) when holistic prediction is used, compared to when a prescribed algorithm is used, in line with earlier research (Nolan et al., 2016). Similarly, the word “positive” reflects higher personal control and higher perceived competence in the following hypotheses.

4. Although the theory of planned behaviour also suggests attitude and control as two other independent determinants of intentions (van der Zee et al., 2002), we solely focused on subjective norms in this study because this antecedent is theoretically most relevant for stakeholder perceptions and decision-makers’ meta-beliefs.