Physiological Indicators of Fluency and Engagement during Sequential and Simultaneous Modes of Human-Robot Collaboration

References

• Abdur-Rahim, J., Morales, Y., Gupta, P., Umata, I., Watanabe, A., Even, J., Suyama, T., & Ishii, S. (2016). Multi-sensor based state prediction for personal mobility vehicles. PLOS One, 11(10), e0162593. https://doi.org/10.1371/journal.pone.0162593
   PubMed Web of Science ®Google Scholar
• Akselrod, S., Gordon, D., Ubel, F. A., Shannon, D. C., Berger, A. C., & Cohen, R. J. (1981). Power spectrum analysis of heart rate fluctuation: A quantitative probe of beat-to-beat cardiovascular control. Science, 213(4504), 220–222. https://doi.org/10.1126/science.6166045
   PubMed Web of Science ®Google Scholar
• Aly, A., Griffiths, S., & Stramandinoli, F. (2017). Metrics and benchmarks in human-robot interaction: Recent advances in cognitive robotics. Cognitive Systems Research, 43, 313–323. https://doi.org/10.1016/j.cogsys.2016.06.002
   Web of Science ®Google Scholar
• Arai, T., Kato, R., & Fujita, M. (2010). Assessment of operator stress induced by robot collaboration in assembly. CIRP Annals, 59(1), 5–8. https://doi.org/10.1016/j.cirp.2010.03.043
   Web of Science ®Google Scholar
• Aube, C., & Rousseau, V. (2005). Team goal commitment and team effectiveness: The role of task interdependence and supportive behaviors. Group Dynamics: Theory, Research, and Practice, 9(3), 189–204. https://doi.org/10.1037/1089-2699.9.3.189
   Web of Science ®Google Scholar
• Baumert, M., Czippelova, B., Ganesan, A., Schmidt, M., Zaunseder, S., & Javorka, M. (2014). Entropy analysis of RR and QT interval variability during orthostatic and mental stress in healthy subjects. Entropy, 16(12), 6384–6393. https://doi.org/10.3390/e16126384
   Web of Science ®Google Scholar
• Berntson, G. G., & Cacioppo, J. T. (1999). Heart rate variability: A neuroscientific perspective for further studies. Cardiac Electrophysiology Review, 3(4), 279–282. https://doi.org/10.1023/A:1009920002142
   Google Scholar
• Berntson, G. G., Cacioppo, J. T., & Grossman, P. (2007). Whither vagal tone. Biological Psychology, 74(2), 295–300. https://doi.org/10.1016/j.biopsycho.2006.08.006
   PubMed Web of Science ®Google Scholar
• Berntson, G. G., Bigger, J. T., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., Nagaraja, H. N., Porges, S. W., Saul, J. P., Stone, P. H., & van der Molen, M. W. (1997). Heart rate variability: Origins, methods, and ­interpretive caveats. Psychophysiology, 34(6), 623–648. https://doi.org/10.1111/j.1469-8986.1997.tb02140.x
   PubMed Web of Science ®Google Scholar
• Billman, G. E. (2013). The LF/HF ratio does not accurately measure cardiac sympatho-vagal balance. Frontiers in Physiology, 4, 26. https://doi.org/10.3389/fphys.2013.00026
   PubMed Web of Science ®Google Scholar
• Blons, E., Arsac, L. M., Gilfriche, P., McLeod, H., Lespinet-Najib, V., Grivel, E., & Deschodt-Arsac, V. (2019). Alterations in heart-brain interactions under mild stress during a cognitive task are reflected in entropy of heart rate dynamics. Scientific Reports, 9(1), 18190. https://doi.org/10.1038/s41598-019-54547-7
   PubMedGoogle Scholar
• Butler, J. T., & Agah, A. (2001). Psychological effects of ­behavior patterns of a mobile personal robot. Autonomous Robots, 10(2), 185–202. https://doi.org/10.1023/A:1008986004181
   Web of Science ®Google Scholar
• Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making.
   Google Scholar
• Cesta, A., Orlandini, A., Bernardi, G., & Umbrico, A. (2016). Towards a planning-based framework for symbiotic human-robot collaboration. In 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA) (pp. 1–8). https://doi.org/10.1109/ETFA.2016.7733585
   Google Scholar
• Chao, C., & Thomaz, A. L. (2012). Timing in multimodal turn-taking interactions: Control and analysis using timed petri nets.
   Google Scholar
• Choi, S. H., Park, K.-B., Roh, D. H., Lee, J. Y., Mohammed, M., Ghasemi, Y., & Jeong, H. (2022). An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation. Robotics and Computer-Integrated Manufacturing, 73, 102258. https://doi.org/10.1016/j.rcim.2021.102258
   Web of Science ®Google Scholar
• Colgate, E., Bicchi, A., Peshkin, M. A., & Colgate, J. E. (2008). Safety for physical human-robot interaction. In Springer handbook of robotics (pp. 1335–1348). Springer.
   Google Scholar
• Cooke, N. J., Cohen, M. C., Fazio, W. C., Inderberg, L. H., Johnson, C. J., Lematta, G. J., Peel, M., & Teo, A. (2023). From teams to teamness: Future directions in the science of team cognition. Human Factors, 0(0), 187208231162449. https://doi.org/10.1177/00187208231162449
   Google Scholar
• Czyzewska, E., Kiczka, K., Czarnecki, A., & Pokinko, P. (1983). The surgeon’s mental load during decision making at various stages of operations. European Journal of Applied Physiology and Occupational Physiology, 51(3), 441–446. https://doi.org/10.1007/BF00429080
   PubMed Web of Science ®Google Scholar
• De Visser, E. J., Pak, R., & Shaw, T. H. (2018). From ‘automation’ to ‘autonomy’: The importance of trust repair in human–machine interaction. Ergonomics, 61(10), 1409–1427. https://doi.org/10.1080/00140139.2018.1457725
   PubMed Web of Science ®Google Scholar
• Delgado-Bonal, A., & Marshak, A. (2019). Approximate entropy and sample entropy: A comprehensive tutorial. Entropy, 21(6), 541. https://doi.org/10.3390/e21060541
   PubMed Web of Science ®Google Scholar
• Delliaux, S., Delaforge, A., Deharo, J.-C., & Chaumet, G. (2019). Mental workload alters heart rate variability, lowering non-linear dynamics. Frontiers in Physiology, 10, 565. https://doi.org/10.3389/fphys.2019.00565
   PubMed Web of Science ®Google Scholar
• Dickerson, S. S., & Kemeny, M. E. (2004). Acute stressors and cortisol responses: A theoretical integration and synthesis of laboratory research. Psychological Bulletin, 130(3), 355–391. https://doi.org/10.1037/0033-2909.130.3.355
   PubMed Web of Science ®Google Scholar
• Dobbins, C., & Fairclough, S. (2019). Signal processing of multimodal mobile lifelogging data towards detecting stress in real-world driving. IEEE Transactions on Mobile Computing, 18(3), 632–644. https://doi.org/10.1109/TMC.2018.2840153
   Web of Science ®Google Scholar
• El Zaatari, S., Marei, M., Li, W., & Usman, Z. (2019). Cobot programming for collaborative industrial tasks: An overview. Robotics and Autonomous Systems, 116, 162–180. https://doi.org/10.1016/j.robot.2019.03.003
   Web of Science ®Google Scholar
• Erebak, S., & Turgut, T. (2019). Caregivers’ attitudes toward potential robot coworkers in elder care. Cognition, Technology & Work, 21(2), 327–336. https://doi.org/10.1007/s10111-018-0512-0
   Web of Science ®Google Scholar
• Faes, L., Gómez-Extremera, M., Pernice, R., Carpena, P., Nollo, G., Porta, A., & Bernaola-Galván, P. (2019). Comparison of methods for the assessment of nonlinearity in short-term heart rate variability under different physiopathological states. Chaos, 29(12), 123114. https://doi.org/10.1063/1.5115506
   PubMed Web of Science ®Google Scholar
• Fratczak, P., Goh, Y. M., Kinnell, P., Justham, L., & Soltoggio, A. (2020). Virtual reality study of human adaptability in industrial human-robot collaboration. In 2020 IEEE International Conference on Human-Machine Systems (ICHMS) (pp. 1–6). https://doi.org/10.1109/ICHMS49158.2020.9209558
   Google Scholar
• Gombolay, M., Bair, A., Huang, C., & Shah, J. (2017). Computational design of mixed-initiative human–robot teaming that considers human factors: Situational awareness, workload, and workflow preferences. The International Journal of Robotics Research, 36(5–7), 597–617. https://doi.org/10.1177/0278364916688255
   Web of Science ®Google Scholar
• Gombolay, M. C., Gutierrez, R. A., Clarke, S. G., Sturla, G. F., & Shah, J. A. (2015). Decision-making authority, team efficiency and human worker satisfaction in mixed human–robot teams. Autonomous Robots, 39(3), 293–312. https://doi.org/10.1007/s10514-015-9457-9
   Web of Science ®Google Scholar
• Gombolay, M. C., Huang, C., & Shah, J. (2015). Coordination of human-robot teaming with human task preferences. In 2015 AAAI Fall Symposium Series.
   Google Scholar
• Hoffman, G. (2019). Evaluating fluency in human–robot collaboration. IEEE Transactions on Human-Machine Systems, 49(3), 209–218. https://doi.org/10.1109/THMS.2019.2904558
   Web of Science ®Google Scholar
• Hoffman, G., & Breazeal, C. (2010). Effects of anticipatory perceptual simulation on practiced human-robot tasks. Autonomous Robots, 28(4), 403–423. https://doi.org/10.1007/s10514-009-9166-3
   Web of Science ®Google Scholar
• Hopko, S. K., & Mehta, R. K. (2022). Trust in shared-space collaborative robots: Shedding light on the human brain. Human Factors, 0(0): 187208221109039. https://doi.org/10.1177/00187208221109039
   Google Scholar
• Hopko, S. K., Mehta, R. K., & Pagilla, P. R. (2023). Physiological and perceptual consequences of trust in collaborative robots: An empirical investigation of human and robot factors. Applied Ergonomics, 106, 103863. https://doi.org/10.1016/j.apergo.2022.103863
   PubMed Web of Science ®Google Scholar
• Hopko, S., Wang, J., & Mehta, R. (2022). Human factors considerations and metrics in shared space human-robot collaboration: A systematic review. Frontiers in Robotics and AI, 9, 799522. https://doi.org/10.3389/frobt.2022.799522
   PubMed Web of Science ®Google Scholar
• Hu, W.-L., Akash, K., Jain, N., & Reid, T. (2016). Real-time sensing of trust in human-machine interactions. IFAC-PapersOnLine, 49(32), 48–53. https://doi.org/10.1016/j.ifacol.2016.12.188
   Google Scholar
• Huang, C.-M., Cakmak, M., & Mutlu, B. (2015). Adaptive coordination strategies for human-robot handovers. Robotics: Science and Systems, 11, 1–10.
   Google Scholar
• Jennings, J. R., Allen, B., Gianaros, P. J., Thayer, J. F., & Manuck, S. B. (2015). Focusing neurovisceral integration: Cognition, heart rate variability, and cerebral blood flow. Psychophysiology, 52(2), 214–224. https://doi.org/10.1111/psyp.12319
   PubMed Web of Science ®Google Scholar
• Keitel, A., Ringleb, M., Schwartges, I., Weik, U., Picker, O., Stockhorst, U., & Deinzer, R. (2011). Endocrine and psychological stress responses in a simulated emergency situation. Psychoneuroendocrinology, 36(1), 98–108. https://doi.org/10.1016/j.psyneuen.2010.06.011
   PubMed Web of Science ®Google Scholar
• Krüger, J., Lien, T. K., & Verl, A. (2009). Cooperation of human and machines in assembly lines. CIRP Annals, 58(2), 628–646. https://doi.org/10.1016/j.cirp.2009.09.009
   Web of Science ®Google Scholar
• Kulic, D., & Croft, E. (2005). Anxiety detection during human-robot interaction. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 616–621). https://doi.org/10.1109/IROS.2005.1545012
   Google Scholar
• Landi, C. T., Villani, V., Ferraguti, F., Sabattini, L., Secchi, C., & Fantuzzi, C. (2018). Relieving operators’ workload: Towards affective robotics in industrial scenarios. Mechatronics, 54, 144–154. https://doi.org/10.1016/j.mechatronics.2018.07.012
   Web of Science ®Google Scholar
• Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
   PubMed Web of Science ®Google Scholar
• Lehrer, P. M. (2007). Biofeedback training to increase heart rate variability. Principles and Practice of Stress Management, 3, 227–248.
   Google Scholar
• Liu, C., Rani, P., & Sarkar, N. (2006). Human-robot interaction using affective cues. In ROMAN 2006-The 15th IEEE International Symposium on Robot and Human Interactive Communication (pp. 285–290).
   Google Scholar
• Lyons, J. B., Sycara, K., Lewis, M., & Capiola, A. (2021). Human–autonomy teaming: Definitions, debates, and directions. Frontiers in Psychology, 12, 589585. https://doi.org/10.3389/fpsyg.2021.589585
   PubMed Web of Science ®Google Scholar
• Malik, M. (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use: Task force of the European Society of Cardiology and the North American Society for Pacing and Electrophysiology. Annals of Noninvasive Electrocardiology, 1(2), 151–181. https://doi.org/10.1111/j.1542-474X.1996.tb00275.x
   Google Scholar
• Maniadakis, M., Hourdakis, E., & Trahanias, P. (2017). Time-informed task planning in multi-agent collaboration. Cognitive Systems Research, 43, 291–300. https://doi.org/10.1016/j.cogsys.2016.09.004
   Web of Science ®Google Scholar
• Matheson, E., Minto, R., Zampieri, E. G. G., Faccio, M., & Rosati, G. (2019). Human–robot collaboration in manufacturing applications: A review. Robotics, 8(4), 100. https://doi.org/10.3390/robotics8040100
   Web of Science ®Google Scholar
• Mohammed, M., Jeong, H., & Lee, J. Y. (2021). Human-robot collision avoidance scheme for industrial settings based on injury classification. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (pp. 549–551). https://doi.org/10.1145/3434074.3447232
   Google Scholar
• Nikolaidis, S., Lasota, P., Ramakrishnan, R., & Shah, J. (2015). Improved human–robot team performance through cross-training, an approach inspired by human team training practices. The International Journal of Robotics Research, 34(14), 1711–1730. https://doi.org/10.1177/0278364915609673
   Web of Science ®Google Scholar
• Nikolaidis, S., & Shah, J. (2012). Human-robot teaming using shared mental models. ACM/IEEE HRI.
   Google Scholar
• Nomura, T., Suzuki, T., Kanda, T., & Kato, K. (2006). Measurement of negative attitudes toward robots. Interaction Studies, 7(3), 437–454. https://doi.org/10.1075/is.7.3.14nom
   Web of Science ®Google Scholar
• Paliga, M., & Pollak, A. (2021). Development and validation of the fluency in human-robot interaction scale. A two-wave study on three perspectives of fluency. International Journal of Human-Computer Studies, 155, 102698. https://doi.org/10.1016/j.ijhcs.2021.102698
   Web of Science ®Google Scholar
• Pham, T., Lau, Z. J., Chen, S. H. A., & Makowski, D. (2021). Heart rate variability in psychology: A review of HRV indices and an analysis tutorial. Sensors, 21(12), 3998. https://doi.org/10.3390/s21123998
   PubMed Web of Science ®Google Scholar
• Ramadurai, S., & Jeong, H. (2022). Effect of human involvement on work performance and fluency in human-robot collaboration for recycling. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 1007–1011). https://doi.org/10.1109/HRI53351.2022.9889606
   Google Scholar
• Rani, P., Sarkar, N., & Adams, J. (2007). Anxiety-based affective communication for implicit human–machine interaction. Advanced Engineering Informatics, 21(3), 323–334. https://doi.org/10.1016/j.aei.2006.11.009
   Web of Science ®Google Scholar
• Roy, R. N., Drougard, N., Gateau, T., Dehais, F., & Chanel, C. P. C. (2020). How can physiological computing benefit human-robot interaction? Robotics, 9(4), 100. https://doi.org/10.3390/robotics9040100
   Web of Science ®Google Scholar
• Shaffer, F., & Ginsberg, J. P. (2017). An overview of heart rate variability metrics and norms. Frontiers in Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
   PubMed Web of Science ®Google Scholar
• Shaffer, F., McCraty, R., & Zerr, C. L. (2014). A healthy heart is not a metronome: An integrative review of the heart’s anatomy and heart rate variability. Frontiers in Psychology, 5, 1040. https://doi.org/10.3389/fpsyg.2014.01040
   PubMed Web of Science ®Google Scholar
• Shah, J., & Breazeal, C. (2010). An empirical analysis of team coordination behaviors and action planning with application to human–robot teaming. Human Factors, 52(2), 234–245. https://doi.org/10.1177/0018720809350882
   PubMed Web of Science ®Google Scholar
• Solís-Montufar, E. E., Gálvez-Coyt, G., & Muñoz-Diosdado, A. (2020). Entropy analysis of RR-time series from stress tests. Frontiers in Physiology, 11, 981. https://doi.org/10.3389/fphys.2020.00981
   PubMed Web of Science ®Google Scholar
• Stout, R. J., Cannon-Bowers, J. A., Salas, E., & Milanovich, D. M. (1999). Planning, shared mental models, and coordinated performance: An empirical link is established. Human Factors, 41(1), 61–71. https://doi.org/10.1518/001872099779577273
   Web of Science ®Google Scholar
• Thayer, J. F., Hansen, A. L., Saus-Rose, E., & Johnsen, B. H. (2009). Heart rate variability, prefrontal neural function, and cognitive performance: The neurovisceral integration perspective on self-regulation, adaptation, and health. Annals of Behavioral Medicine, 37(2), 141–153. https://doi.org/10.1007/s12160-009-9101-z
   PubMed Web of Science ®Google Scholar
• Tobe, F. (2015). Why co-bots will be a huge innovation and growth driver for robotics industry. IEEE Spectrum, 12. https://spectrum.ieee.org/collaborative-robots-innovation-growth-driver
   Google Scholar
• Tokadlı, G., & Dorneich, M. C. (2019). Interaction paradigms: From human-human teaming to human-autonomy teaming. In 2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC) (pp. 1–8). https://doi.org/10.1109/DASC43569.2019.9081665
   Google Scholar
• Villani, V., Pini, F., Leali, F., & Secchi, C. (2018). Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications. Mechatronics, 55, 248–266. https://doi.org/10.1016/j.mechatronics.2018.02.009
   Web of Science ®Google Scholar
• Villani, V., Sabattini, L., Secchi, C., & Fantuzzi, C. (2018). A framework for affect-based natural human-robot interaction. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 1038–1044). https://doi.org/10.1109/ROMAN.2018.8525658
   Google Scholar
• Wynne, K. T., & Lyons, J. B. (2018). An integrative model of autonomous agent teammate-likeness. Theoretical Issues in Ergonomics Science, 19(3), 353–374. https://doi.org/10.1080/1463922X.2016.1260181
   Web of Science ®Google Scholar
• Young, H., & Benton, D. (2015). We should be using nonlinear indices when relating heart-rate dynamics to ­cognition and mood. Scientific Reports, 5(1), 16619. https://doi.org/10.1038/srep16619
   PubMedGoogle Scholar
• Zhao, F., Henrichs, C., & Mutlu, B. (2020). Task interdependence in human-robot teaming. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 1143–1149). https://doi.org/10.1109/RO-MAN47096.2020.9223555
   Google Scholar