Are engineers more likely to avoid algorithms after they see them err? A longitudinal study

ABSTRACT

Research suggests the superior predictive capabilities of algorithms compared to humans. However, people's reluctance to use algorithms after witnessing their inaccuracies has hindered their widespread adoption. Studies have explored this reluctance, but little is known about how different people use algorithms. We focused on algorithm utilisation by engineers, conducting two longitudinal ecological momentary assessment studies outside the lab to explore differences in how engineers and non-engineers engage with inaccurate algorithms. These studies involved 427 participants, predicting currency exchange rates or maximum weather temperatures over nine days based on the judge-advisor system framework. Our results showed a significant three-way interaction between the effects of advice source, whether participants were engineers or non-engineers, and time. Specifically, the trend of inaccurate algorithm use significantly decreased over time for engineers, highlighting the importance of considering the end-users when implementing algorithms.

KEYWORDS:

• Algorithm use
• algorithm acceptance
• engineers’ behaviour
• longitudinal study
• decision-making

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 We followed common practices in the algorithm acceptance literature about considering advice from ‘experts’ (human or algorithmic; Chacon, Kausel, and Reyes 2022; Önkal et al. 2009; Prahl and Van Swol 2017). We expected that people would be willing to improve their predictions with the help of expert knowledge.

Additional information

Funding

This work was partially supported by grants ANID/FONDECYT -Regular/1211367 and ANID/FONDECYT-Regular/1221824.