Are engineers more likely to avoid algorithms after they see them err? A longitudinal study

References

• Abbiati, M., and B. Cerutti. 2023. “Do Students’ Personality Traits Change During Medical Training? A Longitudinal Cohort Study.” Advances in Health Sciences Education, https://doi.org/10.1007/s10459-023-10205-2.
   PubMed Web of Science ®Google Scholar
• Agrawal, A., J. Gans, and A. Goldfarb. 2022. Prediction Machines, Updated and Expanded: The Simple Economics of Artificial Intelligence. Boston, MA: Harvard Business Press.
   Google Scholar
• Aguinis, H., R. K. Gottfredson, and S. A. Culpepper. 2013. “Best-Practice Recommendations for Estimating Cross-Level Interaction Effects Using Multilevel Modeling.” Journal of Management 39 (6): 1490–1528. https://doi.org/10.1177/0149206313478188.
   Web of Science ®Google Scholar
• Ajzen, I. 1991. “The Theory of Planned Behavior.” Organizational Behavior and Human Decision Processes 50 (2): 179–211. https://doi.org/10.1016/0749-5978(91)90020-T.
   Web of Science ®Google Scholar
• Alam, M. Z., N. Nasir, and C. A. Rehman. 2020. “Intrapreneurship Concepts for Engineers: A Systematic Review of the Literature on its Theoretical Foundations and Agenda for Future Research.” Journal of Innovation and Entrepreneurship 9 (1): 1–21. https://doi.org/10.1186/s13731-020-00119-3.
   Google Scholar
• Allen, R. T., and P. Choudhury. 2022. “Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion.” Organization Science 33 (1): 149–169. https://doi.org/10.1287/ORSC.2021.1554.
   Web of Science ®Google Scholar
• Araujo, T., N. Helberger, S. Kruikemeier, and C. H. de Vreese. 2020. “In AI we Trust? Perceptions About Automated Decision-Making by Artificial Intelligence.” AI and Society 35 (3): 611–623. https://doi.org/10.1007/S00146-019-00931-W.
   Web of Science ®Google Scholar
• Berger, B., M. Adam, A. Rühr, and A. Benlian. 2020. “Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn.” Business and Information Systems Engineering, 1–14. https://doi.org/10.1007/s12599-020-00678-5.
   Web of Science ®Google Scholar
• Bielefeldt, A., and G. Rulifson. 2016. “Attitudes that Students Believe Best Characterize Engineers.” ASEE Annual Conference & Exposition. https://peer.asee.org/attitudes-that-students-believe-best-characterize-engineers.
   Google Scholar
• Bigman, Y. E., and K. Gray. 2018. “People are Averse to Machines Making Moral Decisions.” Cognition 181: 21–34. https://doi.org/10.1016/j.cognition.2018.08.003.
   PubMed Web of Science ®Google Scholar
• Bolger, N., and J.-P. Laurenceau. 2013. Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research. New York, NY: Guilford Press.
   Google Scholar
• Bonaccio, S., and R. S. Dalal. 2006. “Advice Taking and Decision-Making: An Integrative Literature Review, and Implications for the Organizational Sciences.” Organizational Behavior and Human Decision Processes 101 (2): 127–151. https://doi.org/10.1016/j.obhdp.2006.07.001.
   Web of Science ®Google Scholar
• Brown, C. L., and D. R. Utley. 2019. “Ambiguity Aversion in Engineers.” Engineering Management Journal 31 (1): 2–7. https://doi.org/10.1080/10429247.2018.1503037.
   Web of Science ®Google Scholar
• Bryan, J. D., and T. Zuva. 2021. “A Review on TAM and TOE Framework Progression and How These Models Integrate.” Advances in Science, Technology and Engineering Systems Journal 6 (3): 137–145. https://doi.org/10.25046/AJ060316.
   Google Scholar
• Burghardt, M. 1995. Introduction to the Engineering Profession. 2nd ed.. New York, NY: HarperCollins.
   Google Scholar
• Burton, J. W., M. Stein, and T. B. Jensen. 2020. “A Systematic Review of Algorithm Aversion in Augmented Decision Making.” Journal of Behavioral Decision Making 33 (2): 220–239. https://doi.org/10.1002/bdm.2155.
   Web of Science ®Google Scholar
• Camilleri, M. A. 2023. “Artificial Intelligence Governance: Ethical Considerations and Implications for Social Responsibility.” Expert Systems, e13406. https://doi.org/10.1111/exsy.13406.
   Web of Science ®Google Scholar
• Capponi, A., S. Ólafsson, and T. Zariphopoulou. 2022. “Personalized Robo-Advising: Enhancing Investment Through Client Interaction.” Management Science 68 (4): 2485–2512. https://doi.org/10.1287/MNSC.2021.4014.
   Web of Science ®Google Scholar
• Carmeli, A., Z. Sheaffer, and M. Y. Halevi. 2009. “Does Participatory Decision-Making in top Management Teams Enhance Decision Effectiveness and Firm Performance?” Personnel Review 38 (6): 696–714. https://doi.org/10.1108/00483480910992283/FULL/XML.
   Web of Science ®Google Scholar
• Carpentier, C. L., and H. Braun. 2020. “Agenda 2030 for Sustainable Development: A Powerful Global Framework.” Journal of the International Council for Small Business 1 (1): 14–23. https://doi.org/10.1080/26437015.2020.1714356.
   Google Scholar
• Carter, J. R., and M. D. Irons. 1991. “Are Economists Different, and If So, Why?” Journal of Economic Perspectives 5 (2): 171–177. https://doi.org/10.1257/jep.5.2.171.
   Web of Science ®Google Scholar
• Castelo, N., M. W. Bos, and D. R. Lehmann. 2019. “Task-dependent Algorithm Aversion.” Journal of Marketing Research 56 (5): 809–825. https://doi.org/10.1177/0022243719851788.
   Web of Science ®Google Scholar
• Chacon, A., E. E. Kausel, and T. Reyes. 2022. “A Longitudinal Approach for Understanding Algorithm use.” Journal of Behavioral Decision Making 35 (4): e2275. https://doi.org/10.1002/bdm.2275.
   Web of Science ®Google Scholar
• Chacon, A., E. E. Kausel, T. Reyes, and S. Trautmann. 2021. “Preventing Algorithm Aversion? People Are Willing to Use Algorithms They Perceive as Learning.” Balas Annual Conference.
   Google Scholar
• Cheng, M., and R. Hackett. 2021. “A Critical Review of Algorithms in HRM: Definition, Theory, and Practice.” Human Resource Management Review 31 (1): 100698. https://doi.org/10.1016/j.hrmr.2019.100698.
   Web of Science ®Google Scholar
• Chong, L., G. Zhang, K. Goucher-Lambert, K. Kotovsky, and J. Cagan. 2022. “Human Confidence in Artificial Intelligence and in Themselves: The Evolution and Impact of Confidence on Adoption of AI Advice.” Computers in Human Behavior 127: 107018. https://doi.org/10.1016/J.CHB.2021.107018.
   Web of Science ®Google Scholar
• Choudhury, P., E. Starr, and R. Agarwal. 2020. “Machine Learning and Human Capital Complementarities: Experimental Evidence on Bias Mitigation.” Strategic Management Journal 41 (8): 1381–1411. https://doi.org/10.1002/SMJ.3152.
   Web of Science ®Google Scholar
• Cormen, T. H., C. E. Leiserson, R. L. Rivest, and C. Stein. 2022. Introduction to Algorithms. Cambridge, MA: MIT press.
   Google Scholar
• Culp, G., and A. Smith. 2009. “Consulting Engineers: Myers-Briggs Type and Temperament Preferences.” Leadership and Management in Engineering 9 (2): 65–70. https://doi.org/10.1061/(ASCE)1532-6748(2009)9:2(65).
   Google Scholar
• Davis, F. D. 1989. “Perceived Usefulness, Perceived Ease of use, and User Acceptance of Information Technology.” MIS Quarterly: Management Information Systems 13 (3): 319–339. https://doi.org/10.2307/249008.
   Web of Science ®Google Scholar
• Dawes, R. M. 1979. “The Robust Beauty of Improper Linear Models in Decision Making.” American Psychologist 34 (7): 571–582. https://doi.org/10.1037/0003-066X.34.7.571.
   Web of Science ®Google Scholar
• Dawson, J. F. 2014. “Moderation in Management Research: What, why, When, and how.” Journal of Business and Psychology 29 (1): 1–19. https://doi.org/10.1007/s10869-013-9308-7.
   Web of Science ®Google Scholar
• Debus, M. E., S. Sonnentag, W. Deutsch, and F. W. Nussbeck. 2014. “Making Flow Happen: The Effects of Being Recovered on Work-Related Flow Between and Within Days.” Journal of Applied Psychology 99 (4): 713–722. https://doi.org/10.1037/a0035881.
   PubMed Web of Science ®Google Scholar
• Dietvorst, B. J., J. P. Simmons, and C. Massey. 2015. “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them err.” Journal of Experimental Psychology: General 144 (1): 114–126. https://doi.org/10.1037/xge0000033.
   PubMed Web of Science ®Google Scholar
• Dietvorst, B. J., J. P. Simmons, and C. Massey. 2018. “Overcoming Algorithm Aversion: People Will use Imperfect Algorithms if They Can (Even Slightly) Modify Them.” Management Science 64 (3): 1155–1170. https://doi.org/10.1287/mnsc.2016.2643.
   Web of Science ®Google Scholar
• Dzindolet, M. T., L. G. Pierce, H. P. Beck, and L. A. Dawe. 2002. “The Perceived Utility of Human and Automated Aids in a Visual Detection Task.” Human Factors: The Journal of the Human Factors and Ergonomics Society 44 (1): 79–94. https://doi.org/10.1518/0018720024494856.
   PubMed Web of Science ®Google Scholar
• Etzioni, A. 2015. “The Moral Effects of Economic Teaching.” Sociological Forum 30 (1): 228–233. https://doi.org/10.1111/socf.12153.
   Web of Science ®Google Scholar
• Fenneman, A., J. Sickmann, T. Pitz, and A. G. Sanfey. 2021. “Two Distinct and Separable Processes Underlie Individual Differences in Algorithm Adherence: Differences in Predictions and Differences in Trust Thresholds.” PLoS One 16 (2): e0247084. https://doi.org/10.1371/JOURNAL.PONE.0247084.
   PubMed Web of Science ®Google Scholar
• Frank, R. H., T. Gilovich, and D. Regan. 1993. “Does Studying Economics Inhibit Cooperation?” Journal of Economic Perspectives 7 (2): 159–171. https://doi.org/10.1515/9781400833917.155.
   Web of Science ®Google Scholar
• Frank, B., and G. G. Schulze. 2000. “Does Economics Make Citizens Corrupt?” Journal of Economic Behavior and Organization 43 (1): 101–113. https://doi.org/10.1016/s0167-2681(00)00111-6.
   Web of Science ®Google Scholar
• Gardner, N. R., J. D. Ritschel, E. D. White, and A. T. Wallen. 2017. “Forecasting Foreign Currency Exchange Rates for Department of Defense Budgeting.” Journal of Public Procurement 17 (3): 315–336. https://doi.org/10.1108/JOPP-17-03-2017-B002.
   Google Scholar
• Germonprez, M., and I. Zigurs. 2009. “Task, Technology, and Tailoring in Communicative Action: An in-Depth Analysis of Group Communication.” Information and Organization 19 (1): 22–46. https://doi.org/10.1016/J.INFOANDORG.2008.03.002.
   Web of Science ®Google Scholar
• Gillespie, T. 2014. “The Relevance of Algorithms.” Media Technologies: Essays on Communication, Materiality, and Society 167. https://doi.org/10.7551/mitpress/9780262525374.001.0001
   Google Scholar
• Gino, F., and D. A. Moore. 2007. “Effects of Task Difficulty on use of Advice.” Journal of Behavioral Decision Making 20 (1): 21–35. https://doi.org/10.1002/bdm.539.
   Web of Science ®Google Scholar
• Goldman, B., D. A. Cooper, and C. Koc. 2019. “An Exploration of Whether Engineers Differ from non-Engineers in Their Approach to Negotiations.” International Journal of Conflict Management 30 (4): 420–440. https://doi.org/10.1108/IJCMA-02-2019-0034/FULL/HTML.
   Web of Science ®Google Scholar
• Greenwood, E. 1957. “Attributes of a Profession.” Social Work 2 (3): 45–55. https://doi.org/10.1093/sw/2.3.45.
   Web of Science ®Google Scholar
• Gridley, M. C. 2007. “Differences in Thinking Styles of Artists and Engineers.” Career Development Quarterly 56 (2): 177–182. https://doi.org/10.1002/J.2161-0045.2007.TB00030.X.
   Web of Science ®Google Scholar
• Grove, W. M., D. H. Zald, B. S. Lebow, B. E. Snitz, and C. Nelson. 2000. “Clinical Versus Mechanical Prediction: A Meta-Analysis.” Psychological Assessment 12 (1): 19–30. https://doi.org/10.1037/1040-3590.12.1.19.
   PubMed Web of Science ®Google Scholar
• Haesevoets, T., D. De Cremer, K. Dierckx, and A. Van Hiel. 2021. “Human-machine Collaboration in Managerial Decision Making.” Computers in Human Behavior 119: 106730. https://doi.org/10.1016/J.CHB.2021.106730.
   Web of Science ®Google Scholar
• Harmon, L. W., D. W. DeWitt, D. P. Campbell, and J. I. C. Hansen. 1994. Strong Interest Inventory: Applications and Technical Guide: Form T317 of the Strong Vocational Interest Blanks. Stanford University Press.
   Google Scholar
• Harris, J. 1994. “Perceptions of Engineering, Nursing, and Psychology Students’ Personalities.” Canadian Journal of Behavioural Science/Revue Canadienne Des Sciences Du Comportement 26 (4): 484. https://psycnet.apa.org/record/1995-23201-001.
   Web of Science ®Google Scholar
• Haumer, F., L. Schlicker, P. C. Murschetz, and C. Kolo. 2021. “Tailor the Message and Change Will Happen? An Experimental Study of Message Tailoring as an Effective Communication Strategy for Organizational Change.” Journal of Strategy and Management 14 (4): 426–443. https://doi.org/10.1108/JSMA-08-2020-0207.
   Web of Science ®Google Scholar
• Ho, G., D. Wheatley, and C. T. Scialfa. 2005. “Age Differences in Trust and Reliance of a Medication Management System.” Interacting with Computers 17 (6): 690–710. https://doi.org/10.1016/J.INTCOM.2005.09.007.
   Web of Science ®Google Scholar
• Holcomb, T. R., J. G. Combs, D. G. Sirmon, and J. Sexton. 2010. “Modeling Levels and Time in Entrepreneurship Research.” Organizational Research Methods 13 (2): 348–389. https://doi.org/10.1177/1094428109338401.
   Web of Science ®Google Scholar
• Howell, L. L., C. D. Sorensen, and M. R. Jones. 2014. “Are Undergraduate GPA and General GRE Percentiles Valid Predictors of Student Performance in an Engineering Graduate Program?” International Journal of Engineering Education 30 (5): 1145–1165.
   Web of Science ®Google Scholar
• Hox, J. J., M. Moerbeek, and R. Van de Schoot. 2017. Multilevel Analysis: Techniques and Applications. New York, NY: Routledge.
   Google Scholar
• Jensen, J. 2006. A User’s Guide to Engineering. Pearson: Prentice Hall.
   Google Scholar
• Jiang, X. Y., X. C. Huang, J. P. Huang, and Y. F. Tong. 2022. “Real-Time Intelligent Elevator Monitoring and Diagnosis: Case Studies and Solutions with Applications using Artificial Intelligence.” Computers and Electrical Engineering 100: 107965. https://doi.org/10.1016/J.COMPELECENG.2022.107965.
   Web of Science ®Google Scholar
• Kaufmann, E., A. Chacon, E. Kausel, N. Herrera, and T. Reyes. 2023. “Task-specific Algorithm Advice Acceptance: A Review and Directions for Future Research.” Data and Information Management 7 (3): 100040. https://doi.org/10.1016/J.DIM.2023.100040.
   Google Scholar
• Kawaguchi, K. 2020. “When Will Workers Follow an Algorithm? A Field Experiment with a Retail Business.” Management Science 67 (3): 1670–1695. https://doi.org/10.1287/mnsc.2020.3599.
   Web of Science ®Google Scholar
• Kellogg, K., M. Valentine, and A. Christin. 2020. “Algorithms at Work: The new Contested Terrain of Control.” Academy of Management Annals 14 (1): 366–410. https://doi.org/10.5465/annals.2018.0174.
   Web of Science ®Google Scholar
• Komiak, S. Y., and I. Benbasat. 2006. “The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents.” MIS Quarterly: Management Information Systems 30 (4): 941–960. https://doi.org/10.2307/25148760.
   Web of Science ®Google Scholar
• Kreitz, G., and F. Niemelä. 2010. “Spotify - Large Scale, low Latency, P2P Music-on-Demand Streaming.” 2010 IEEE 10th International Conference on Peer-to-Peer Computing, P2P 2010 - Proceedings, 1–10. https://doi.org/10.1109/P2P.2010.5569963.
   Google Scholar
• Langer, M., and R. N. Landers. 2021. “The Future of Artificial Intelligence at Work: A Review on Effects of Decision Automation and Augmentation on Workers Targeted by Algorithms and Third-Party Observers.” Computers in Human Behavior 123: 106878. https://doi.org/10.1016/J.CHB.2021.106878.
   Web of Science ®Google Scholar
• Leary, M. R., and R. H. Hoyle, eds. 2009. Handbook of Individual Differences in Social Behavior. New York, NY: Guilford Press.
   Google Scholar
• Lee, K., H. G. Woo, W. Cho, and S. B. de Jong. 2022. “When Can AI Reduce Individuals’ Anchoring Bias and Enhance Decision Accuracy? Evidence from Multiple Longitudinal Experiments.” Proceedings of the 55th Hawaii International Conference on System Sciences, 2174–2183. https://doi.org/10.24251/hicss.2022.273.
   Google Scholar
• Li, Z., P. Rau, and D. Huang. 2020. “Who Should Provide Clothing Recommendation Services: Artificial Intelligence or Human Experts?” Journal of Information Technology Research (JITR) 13 (3): 113–125. https://doi.org/10.4018/JITR.2020070107.
   Web of Science ®Google Scholar
• Logg, J. M. 2016. When do People Rely on Algorithms? Berkeley: University of California.
   Google Scholar
• Logg, J. M., J. A. Minson, and D. A. Moore. 2019. “Algorithm Appreciation: People Prefer Algorithmic to Human Judgment.” Organizational Behavior and Human Decision Processes 151: 90–103. https://doi.org/10.1016/j.obhdp.2018.12.005.
   Web of Science ®Google Scholar
• Loi, R., N. Hang-yue, and S. Foley. 2004. “The Effect of Professional Identification On Job Attitudes: A Study of Lawyers in Hong Kong.” Organizational Analysis 12 (2): 109–128. https://doi.org/10.1108/EB028988/FULL/XML.
   Google Scholar
• Lourenço, C. J. S., B. G. C. Dellaert, and B. Donkers. 2020. “Whose Algorithm Says So: The Relationships Between Type of Firm, Perceptions of Trust and Expertise, and the Acceptance of Financial Robo-Advice.” Journal of Interactive Marketing 49 (1): 107–124. https://doi.org/10.1016/J.INTMAR.2019.10.003/ASSET/IMAGES/LARGE/10.1016_J.INTMAR.2019.10.003-FIG2.JPEG.
   Google Scholar
• Madhavan, P., and D. A. Wiegmann. 2007. “Effects of Information Source, Pedigree, and Reliability on Operator Interaction with Decision Support Systems.” Human Factors 49 (5): 773–785. https://doi.org/10.1518/001872007X230154.
   PubMed Web of Science ®Google Scholar
• Mahmud, H., A. N. Islam, S. I. Ahmed, and K. Smolander. 2022. “What Influences Algorithmic Decision-Making? A Systematic Literature Review on Algorithm Aversion.” Technological Forecasting and Social Change 175), https://doi.org/10.1016/j.techfore.2021.121390.
   Web of Science ®Google Scholar
• Mahmud, H., A. N. Islam, and R. K. Mitra. 2023. “What Drives Managers Towards Algorithm Aversion and how to Overcome it? Mitigating the Impact of Innovation Resistance Through Technology Readiness.” Technological Forecasting and Social Change 193: 122641. https://doi.org/10.1016/j.techfore.2023.122641.
   Web of Science ®Google Scholar
• Malek, M. A. 2022. “Criminal Courts’ Artificial Intelligence: The way it Reinforces Bias and Discrimination.” AI and Ethics 2 (1): 233–245. https://doi.org/10.1007/S43681-022-00137-9.
   Google Scholar
• Martin, D. A., E. Conlon, and B. Bowe. 2021. “Using Case Studies in Engineering Ethics Education: The Case for Immersive Scenarios Through Stakeholder Engagement and Real Life Data.” Australasian Journal of Engineering Education 26 (1): 47–63. https://doi.org/10.1080/22054952.2021.1914297.
   Google Scholar
• Mathieu, J. E., H. Aguinis, S. A. Culpepper, and G. Chen. 2012. “Understanding and Estimating the Power to Detect Cross-Level Interaction Effects in Multilevel Modeling.” Journal of Applied Psychology 97 (5): 951–966. https://doi.org/10.1037/a0028380.
   PubMed Web of Science ®Google Scholar
• Moosa, I. 2013. “Why is it so Difficult to Outperform the Random Walk in Exchange Rate Forecasting?” Applied Economics 45 (23): 3340–3346. https://doi.org/10.1080/00036846.2012.709605.
   Web of Science ®Google Scholar
• Morewedge, C. K., S. Mullainathan, H. F. Naushan, C. R. Sunstein, J. Kleinberg, M. Raghavan, and J. O. Ludwig. 2023. “Human Bias in Algorithm Design.” Nature Human Behaviour 7 (11): 1822–1824. https://doi.org/10.1038/s41562-023-01724-4.
   PubMedGoogle Scholar
• National_Academy_of_Engineering. 2004. The Engineer of 2020: Visions of Engineering in the New Century. Washington, DC: National Academies Press.
   Google Scholar
• Önkal, D., M. S. Gönül, and S. De Baets. 2019. “Trusting Forecasts.” Futures & Foresight Science 1 (3-4): e19. https://doi.org/10.1002/ffo2.19.
   Google Scholar
• Önkal, D., P. Goodwin, M. Thomson, S. Gönül, and A. Pollock. 2009. “The Relative Influence of Advice from Human Experts and Statistical Methods on Forecast Adjustments.” Journal of Behavioral Decision Making 22 (4): 390–409. https://doi.org/10.1002/bdm.637.
   Web of Science ®Google Scholar
• Pawley, A. L. 2009. “Universalized Narratives: Patterns in how Faculty Members Define “Engineering”.” Journal of Engineering Education 98 (4): 309–319. https://doi.org/10.1002/j.2168-9830.2009.tb01029.x.
   Web of Science ®Google Scholar
• Prahl, A., and L. Van Swol. 2017. “Understanding Algorithm Aversion: When is Advice from Automation Discounted?” Journal of Forecasting 36 (6): 691–702. https://doi.org/10.1002/for.2464.
   Web of Science ®Google Scholar
• Preacher, K. J., P. J. Curran, and D. J. Bauer. 2006. “Computational Tools for Probing Interactions in Multiple Linear Regression, Multilevel Modeling, and Latent Curve Analysis.” Journal of Educational and Behavioral Statistics 31 (4): 437–448. https://doi.org/10.3102/10769986031004437.
   Web of Science ®Google Scholar
• Pyster, A., N. Hutchison, and D. Henry. 2018. The Paradoxical Mindset of Systems Engineers: Uncommon Minds, Skills, and Careers. Hoboken, NJ: John Wiley & Sons.
   Google Scholar
• Renier, L., M. Mast, and A. Bekbergenova. 2021. “To err is Human, not Algorithmic–Robust Reactions to Erring Algorithms.” Computers in Human Behavior 124: 106879. https://doi.org/10.1016/j.chb.2021.106879.
   Web of Science ®Google Scholar
• Robinson, M. A. 2010. “An Empirical Analysis of Engineers’ Information Behaviors.” Journal of the American Society for Information Science and Technology 61 (4): 640–658. https://doi.org/10.1002/ASI.21290.
   Web of Science ®Google Scholar
• Shanteau, J., and D. J. Weiss. 2014. “Individual Expertise Versus Domain Expertise.” American Psychologist 69 (7): 711–712. https://doi.org/10.1037/a0037874.
   PubMed Web of Science ®Google Scholar
• Sniezek, J. A., and L. M. Van Swol. 2001. “Trust, Confidence, and Expertise in a Judge-Advisor System.” Organizational Behavior and Human Decision Processes 84 (2): 288–307. https://doi.org/10.1006/OBHD.2000.2926.
   PubMed Web of Science ®Google Scholar
• StataCorp, L. P. 2013. Stata Multilevel Mixed-Effects Reference Manual (Vol. 9, Issue 10). College Station, TX: StataCorp LP.
   Google Scholar
• Sutherland, S. C., C. Harteveld, and M. E. Young. 2016. “Effects of the Advisor and Environment on Requesting and Complying with Automated Advice.” ACM Transactions on Interactive Intelligent Systems 6 (4), https://doi.org/10.1145/2905370.
   PubMedGoogle Scholar
• Thurman, N., J. Moeller, N. Helberger, and D. Trilling. 2019. “My Friends, Editors, Algorithms, and I: Examining Audience Attitudes to News Selection.” Digital Journalism 7 (4): 447–469. https://doi.org/10.1080/21670811.2018.1493936.
   Web of Science ®Google Scholar
• UNESCO, I. for S. 2012. “International Standard Classification of Education: (ISCED) 2011.” Comparative Social Research 30), https://doi.org/10.15220/978-92-9189-123-8-EN.
   Google Scholar
• Van_Berkel, N., D. Ferreira, and V. Kostakos. 2017. “The Experience Sampling Method on Mobile Devices.” ACM Computing Surveys 50 (6): 1–40. https://doi.org/10.1145/3123988.
   Web of Science ®Google Scholar
• Van_Der_Molen, H. T., H. G. Schmidt, and G. Kruisman. 2007. “Personality Characteristics of Engineers.” International Journal of Phytoremediation 32 (5): 495–501. https://doi.org/10.1080/03043790701433111.
   Google Scholar
• Venkatesh, V., M. G. Morris, G. B. Davis, and F. D. Davis. 2003. “User Acceptance of Information Technology: Toward a Unified View.” Management Information Systems (MIS) Quarterly 27 (3): 425–478. https://doi.org/10.2307/30036540.
   Web of Science ®Google Scholar
• Wan, J., Q. Sun, X. Li, J. Ding, and Q. Zhu. 2018. “Personalized Professional Recommendation System Based on Undergraduate Questionnaires.” Proceedings - 2018 17th International Symposium on Distributed Computing and Applications for Business Engineering and Science, DCABES 2018, 140–143. https://doi.org/10.1109/DCABES.2018.00045.
   Google Scholar
• Weiss, D. J., and J. Shanteau. 2003. “Empirical Assessment of Expertise.” Human Factors 45 (1): 104–114. https://doi.org/10.1518/hfes.45.1.104.27233.
   PubMed Web of Science ®Google Scholar
• Williamson, J. M., J. W. Lounsbury, and L. D. Han. 2013. “Key Personality Traits of Engineers for Innovation and Technology Development.” Journal of Engineering and Technology Management 30 (2): 157–168. https://doi.org/10.1016/J.JENGTECMAN.2013.01.003.
   Web of Science ®Google Scholar
• Yaniv, I. 2004. “Receiving Other People’s Advice: Influence and Benefit.” Organizational Behavior and Human Decision Processes 93 (1): 1–13. https://doi.org/10.1016/j.obhdp.2003.08.002.
   Web of Science ®Google Scholar
• Zakrzewska, D. 2009. “Cluster Analysis in Personalized e-Learning Systems.” Studies in Computational Intelligence 252: 229–250. https://doi.org/10.1007/978-3-642-04170-9_10/COVER.
   Google Scholar
• Zhang, L., X. W. Meng, J. L. Chen, K. Duan, and Y. Peng. 2009. “Personalized Service Recommendation Algorithm.” Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009, 522–526. https://doi.org/10.1109/ICCSIT.2009.5234516.
   Google Scholar