Advanced search
Publication Cover
European Journal of Work and Organizational Psychology Volume 33, 2024 - Issue 3
Submit an article Journal homepage
1,026
Views
0
CrossRef citations to date
0
Altmetric
Research Article

“Adding an egg” in algorithmic decision making: improving stakeholder and user perceptions, and predictive validity by enhancing autonomy

Marvin Neumanna Department of Experimental and Applied Psychology, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The NetherlandsCorrespondence[email protected]
ORCID Iconhttps://orcid.org/0000-0003-0193-8159
ORCID Icon,
A. Susan M. Niessenb Department of Psychometrics and Statistics, Faculty of Behavioral and Social Sciences, University of Groningen, Groningen, The NetherlandsORCID Iconhttps://orcid.org/0000-0001-8249-9295
ORCID Icon,
Maximilian Lindeb Department of Psychometrics and Statistics, Faculty of Behavioral and Social Sciences, University of Groningen, Groningen, The NetherlandsORCID Iconhttps://orcid.org/0000-0001-8421-090X
ORCID Icon,
Jorge N. Tendeiroc Office of Research and Academia-Government-Community Collaboration, Education, Research Center for Artificial Intelligence and Data Innovation, Hiroshima University, Hiroshima, JapanORCID Iconhttps://orcid.org/0000-0003-1660-3642
ORCID Icon &
Rob R. Meijerb Department of Psychometrics and Statistics, Faculty of Behavioral and Social Sciences, University of Groningen, Groningen, The NetherlandsORCID Iconhttps://orcid.org/0000-0001-5368-992X
ORCID Icon
Pages 245-262 | Received 23 Mar 2023, Accepted 13 Sep 2023, Published online: 26 Sep 2023

References

  • Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., & Rush, J. D. (2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382. https://doi.org/10.1177/0011000005285875
  • Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T
  • Arkes, H. R., Shaffer, V. A., & Medow, M. A. (2007). Patients derogate physicians who use a computer-assisted diagnostic aid. Medical Decision Making, 27(2), 189–202. https://doi.org/10.1177/0272989X06297391
  • Balzer, W. K., Doherty, M. E., & O’Connor, R., Jr. (1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106(3), 410–433. https://doi.org/10.1037/0033-2909.106.3.410
  • Burton, J. W., Stein, M. K., & Jensen, T. B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239. https://doi.org/10.1002/bdm.2155
  • Castelo, N., Bos, M. W., & Lehmann, D. R. (2019). Task-dependent algorithm aversion. Journal of Marketing Research, 56(5), 809–825. https://doi.org/10.1177/0022243719851788
  • Cortina, J. M., Goldstein, N. B., Payne, S. C., Davison, H. K., & Gilliland, S. W. (2000). The incremental validity of interview scores over and above cognitive ability and conscientiousness scores. Personnel Psychology, 53(2), 325–351. https://doi.org/10.1111/j.1744-6570.2000.tb00204.x
  • Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81(2), 95–106. https://doi.org/10.1037/h0037613
  • Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573
  • Diab, D. L., Pui, S., Yankelevich, M., & Highhouse, S. (2011). Lay perceptions of selection decision aids in US and non-US samples. International Journal of Selection and Assessment, 19(2), 209–216. https://doi.org/10.1111/j.1468-2389.2011.00548.x
  • Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155–1170. https://doi.org/10.1287/mnsc.2016.2643
  • Frey, F. E., & Tropp, L. R. (2006). Being seen as individuals versus as group members: Extending research on metaperception to intergroup contexts. Personality and Social Psychology Review, 10(3), 265–280. https://doi.org/10.1207/s15327957pspr1003_5
  • Gonzalez, M. F., Liu, W., Shirase, L., Tomczak, D. L., Lobbe, C. E., Justenhoven, R., & Martin, N. R. (2022). Allying with AI? Reactions toward human-based, AI/ML-based, and augmented hiring processes. Computers in Human Behavior, 130, 107179. https://doi.org/10.1016/j.chb.2022.107179
  • Grömping, U. (2006). Relative importance for linear regression in R: The package relaimpo. Journal of Statistical Software, 17(1), 1–27. https://doi.org/10.18637/jss.v017.i01
  • Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-3590.12.1.19
  • Guay, J. P., & Parent, G. (2018). Broken legs, clinical overrides, and recidivism risk: An analysis of decisions to adjust risk levels with the LS/CMI. Criminal Justice and Behavior, 45(1), 82–100. https://doi.org/10.1177/0093854817719482
  • Hammond, K. R., & Summers, D. A. (1972). Cognitive control. Psychological Review, 79(1), 58–67. https://doi.org/10.1037/h0031851
  • Han, J., Pei, J., & Kamber, M. (2011). Data mining: Concepts and techniques: Vol. 3rd ed. Morgan Kaufmann.
  • Highhouse, S., & Brooks, M. E. (2023). Improving workplace judgments by reducing noise: Lessons learned from a century of selection research. Annual Review of Organizational Psychology and Organizational Behavior, 10(1), 519–533. https://doi.org/10.1146/annurev-orgpsych-120920-050708
  • Hoffman, M., Kahn, L. B., & Li, D. (2017). Discretion in hiring. The Quarterly Journal of Economics, 133(2), 765–800. https://doi.org/10.1093/qje/qjx042
  • Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515–526. https://doi.org/10.1037/a0016755
  • Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: A flaw in human judgment. Little, Brown Spark.
  • Kares, F., König, C. J., Bergs, R., Protzel, C., & Langer, M. (2023). Trust in hybrid human-automated decision-support. International Journal of Selection and Assessment, 31(3), 388–402. https://doi.org/10.1111/ijsa.12423
  • Kausel, E. E., Culbertson, S. S., & Madrid, H. P. (2016). Overconfidence in personnel selection: When and why unstructured interview information can hurt hiring decisions. Organizational Behavior and Human Decision Processes, 137, 27–44. https://doi.org/10.1016/j.obhdp.2016.07.005
  • Kelley, H. H. (1973). The processes of causal attribution. American Psychologist, 28(2), 107–128. https://doi.org/10.1037/h0034225
  • Kruschke, J. K. (2015). Doing bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Academic Press. https://doi.org/10.1016/B978-0-12-405888-0.00008-8
  • Kuncel, N. R. (2018). Judgment and decision making in staffing research and practice. In D. S. Ones, N. Anderson, C. Viswesvaran, & H. K. Sinangil (Eds.), The sage handbook of industrial, work and organizational psychology (2nd ed., pp. 474–487). SAGE Publications Ltd. https://doi.org/10.4135/9781473914940.n17
  • Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis. Journal of Applied Psychology, 98(6), 1060–1072. https://doi.org/10.1037/a0034156
  • Langer, M., & Landers, R. N. (2021). The future of artificial intelligence at work: A review on effects of decision automation and augmentation on workers targeted by algorithms and third-party observers. Computers in Human Behavior, 123, 106878. https://doi.org/10.1016/J.CHB.2021.106878
  • Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. University of Minnesota Press. https://doi.org/10.1037/11281-000
  • Meehl, P. E. (1986). Causes and effects of my disturbing little book. Journal of Personality Assessment, 50(3), 370–375. https://doi.org/10.1207/s15327752jpa5003_6
  • Meijer, R. R., Neumann, M., Hemker, B. T., & Niessen, A. S. M. (2020). A tutorial on mechanical decision-making for personnel and educational selection. Frontiers in Psychology, 10. Article 3002. https://doi.org/10.3389/fpsyg.2019.03002
  • Meijer, R. R., & Niessen, A. S. M. (2022). Personnel selection as judgment and decision science: An introduction. International Journal of Selection and Assessment, 30(2), 193–194. https://doi.org/10.1111/IJSA.12377
  • Meyer, J. P., & Koelbl, S. L. M. (1982). Students’ test performances: Dimensionality of causal attributions. Personality and Social Psychology Bulletin, 8(1), 31–36. https://doi.org/10.1177/014616728281005
  • Mikkelson, D. (2008, January 31). Requiring an Egg Made Instant Cake Mixes Sell? Snopes. https://www.snopes.com/fact-check/something-eggstra/
  • Milkman, K. L., Chugh, D., & Bazerman, M. H. (2009). How can decision making be improved? Perspectives on Psychological Science, 4(4), 379–383. https://doi.org/10.1111/j.1745-6924.2009.01142.x
  • Murphy, K. R. (2019). Understanding how and why adding valid predictors can decrease the validity of selection composites: A generalization of Sackett, Dahlke, shewach, and Kuncel (2017). International Journal of Selection and Assessment, 27(3), 249–255. https://doi.org/10.1111/ijsa.12253
  • Murphy, K. R. (2021). In praise of table 1: The importance of making better use of descriptive statistics. Industrial and Organizational Psychology: Perspectives on Science and Practice, 14(4), 461–477. https://doi.org/10.1017/iop.2021.90
  • Neumann, M., Hengeveld, M., Niessen, A. S. M., Tendeiro, J. N., & Meijer, R. R. (2022). Education increases decision-rule use: An investigation of education and incentives to improve decision making. Journal of Experimental Psychology: Applied, 28(1), 166–178. https://doi.org/10.1037/xap0000372
  • Neumann, M., Niessen, A. S. M., Hurks, P. M., & Meijer, R. R. (2023). Holistic and mechanical combination in psychological assessment: Why algorithms are underutilized and what is needed to increase their use. International Journal of Selection and Assessment, 31(2), 267–285. https://doi.org/10.1111/ijsa.12416
  • Neumann, M., Niessen, A. S. M., & Meijer, R. R. (2021). Implementing evidence-based assessment and selection in organizations: A review and an agenda for future research. Organizational Psychology Review, 11(3), 205–239. https://doi.org/10.1177/2041386620983419
  • Neumann, M., Niessen, A. S. M., Tendeiro, J. N., & Meijer, R. R. (2022). The autonomy-validity dilemma in mechanical prediction procedures: The quest for a compromise. Journal of Behavioral Decision Making, 35(4), e2270. https://doi.org/10.1002/bdm.2270
  • Newman, D. T., Fast, N. J., & Harmon, D. J. (2020). When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions. Organizational Behavior and Human Decision Processes, 160, 149–167. https://doi.org/10.1016/j.obhdp.2020.03.008
  • Niessen, A. S. M., Kausel, E. E., & Neumann, M. (2022). Using narratives and numbers in performance prediction: Attitudes, confidence, and validity. International Journal of Selection and Assessment, 30(2), 216–229. https://doi.org/10.1111/IJSA.12364
  • Nolan, K. P. (2013). Basic psychological need fulfillment and user resistance to objective and analytical decision-making practices in employee selection ( Doctoral dissertation).
  • Nolan, K. P., Carter, N. T., & Dalal, D. K. (2016). Threat of technological unemployment: Are hiring managers discounted for using standardized employee selection practices? Personnel Assessment and Decisions, 2(1), 30–47. https://doi.org/10.25035/pad.2016.004
  • Nolan, K. P., Dalal, D. K., & Carter, N. (2020). Threat of technological unemployment, use intentions, and the promotion of structured interviews in personnel selection. Personnel Assessment and Decisions, 6(2), 38–53. https://doi.org/10.25035/pad.2020.02.006
  • Nolan, K. P., & Highhouse, S. (2014). Need for autonomy and resistance to standardized employee selection practices. Human Performance, 27(4), 328–346. https://doi.org/10.1080/08959285.2014.929691
  • Preacher, K. J., & Kelley, K. (2011). Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods, 16(2), 93–115. https://doi.org/10.1037/a0022658
  • Ryan, A. M., & Sackett, P. R. (1987). A survey of individual assessment practices by I/O psychologists. Personnel Psychology, 40(3), 455–488. https://doi.org/10.1111/j.1744-6570.1987.tb00610.x
  • Sackett, P. R., Dahlke, J. A., Shewach, O. R., & Kuncel, N. R. (2017). Effects of predictor weighting methods on incremental validity. Journal of Applied Psychology, 102(10), 1421–1434. https://doi.org/10.1037/apl0000235
  • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262
  • Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34(2), 125–137. https://doi.org/10.1007/s10869-018-09613-8
  • van der Zee, K. I., Bakker, A. B., & Bakker, P. (2002). Why are structured interviews so rarely used in personnel selection? Journal of Applied Psychology, 87(1), 176–184. https://doi.org/10.1037/0021-9010.87.1.176
  • Wesche, J. S., Hennig, F., Kollhed, C. S., Quade, J., Kluge, S., & Sonderegger, A. (2022). People’s reactions to decisions by human vs. algorithmic decision-makers: The role of explanations and type of selection tests. European Journal of Work and Organizational Psychology, 1–12. https://doi.org/10.1080/1359432X.2022.2132940
  • Yu, M. C., & Kuncel, N. R. (2020). Pushing the limits for judgmental consistency: Comparing random weighting schemes with expert judgments. Personnel Assessment and Decisions, 6(2), 1–10. https://doi.org/10.25035/pad.2020.02.002
  • Yu, M. C., & Kuncel, N. R. (2022). Testing the value of expert insight: Comparing local versus general expert judgment models. International Journal of Selection and Assessment, 30(2), 202–215. https://doi.org/10.1111/IJSA.12356

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.