Multimodal job interview simulator for training of autistic individuals

References

• Adiani, D., Itzkovitz, A., Bian, D., Katz, H., Breen, M., Hunt, S., Swanson, A., Vogus, T. J., Wade, J., & Sarkar, N. (2022, March). Career Interview Readiness in Virtual Reality (CIRVR): A platform for simulated interview training for autistic individuals and their employers. ACM Transactions on Accessible Computing, 15(1), 28. https://doi.org/10.1145/3505560
   Web of Science ®Google Scholar
• American Psychiatric Association. (2013). Diagnostic and Statistical Manual of mental disorders: DSM-5 (5th ed.). https://doi.org/10.1176/appi.books.9780890425596
   Google Scholar
• Baldwin, S., Costley, D., & Warren, A. (2014). Employment activities and experiences of adults with high-functioning autism and asperger’s disorder. Journal of Autism and Developmental Disorders, 44(10), 2440–2449. https://doi.org/10.1007/s10803-014-2112-z
   PubMed Web of Science ®Google Scholar
• Baur, T., Damian, I., Gebhard, P., Porayska-Pomsta, K., & Andr´e, E. (2013). A job interview simulation: social cue-based interaction with a virtual character. 2013 International Conference on Social Computing (pp. 220–227). https://doi.org/10.1109/SocialCom.2013.39
   Google Scholar
• Bian, D., Wade, J., Swanson, A., Warren, Z., & Sarkar, N. (2015). Physiology-based affect recognition during driving in virtual environment for autism intervention. Proceedings of the 2nd International Conference on Physiological Computing Systems (pp. 137–145). https://doi.org/10.5220/0005331301370145
   Google Scholar
• Black, M. H., Mahdi, S., Milbourn, B., Scott, M., Gerber, A., Esposito, C., Falkmer, M., Lerner, M. D., Halladay, A., Ström, E., D’Angelo, A., Falkmer, T., Bölte, S., & Girdler, S. (2020). Multi‐informant international perspectives on the facilitators and barriers to employment for autistic adults. Autism Research, 13(7), 1195–1214. https://doi.org/10.1002/aur.2288
   PubMed Web of Science ®Google Scholar
• Booth, J. (2016). Autism equality in the workplace. removing barriers and challenging discrimination. Jessica Kingsley Publishers.
   Google Scholar
• Bublitz, D. J., Fitzgerald, K., Alarcon, M., D’onofrio, J., & Gillespie-Lynch, K. (2017). Verbal behaviors during employment interviews of college students with and without ASD. Journal of Vocational Rehabilitation, 47(1), 79–92. https://doi.org/10.3233/JVR-170884
   Google Scholar
• Burke, S. L., Bresnahan, T. L., Li, T., Epnere, K., Rizzo, A., Partin, M. T., Ahlness, R. M., & Trimmer, M. (2018). Using virtual interactive training agents (ViTA) with adults with autism and other developmental disabilities. Journal of Autism and Developmental Disorders, 48(3), 905–912. https://doi.org/10.1007/s10803-017-3374-z
   PubMed Web of Science ®Google Scholar
• Ca˜nigueral, R., & Hamilton, A. F. D. C. (2019). The role of eye gaze during natural social interactions in typical and autistic people. Frontiers in Psychology, 10, 10. https://doi.org/10.3389/fpsyg.2019.00560
   PubMed Web of Science ®Google Scholar
• Cacioppo, J. T., Tassinary, L. G., & Berntson, G. (2007). Handbook of Psychophysiology. Cambridge university press.
   Google Scholar
• Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42(3), 294–320. https://doi.org/10.1177/0049124113500475
   Web of Science ®Google Scholar
• Chakravarty, S., Chava, R. V. S. P., & Fox, E. A. (2019). Dialog acts classification for question-answer corpora. ASAIL@ICAIL.
   Google Scholar
• Chang, H. Y., Saleh, M. C., Bruyere, S. M., & Vogus, T. J. (In Press). Making the employment interview work for a neurodiverse workforce: Perspectives of individuals on the autism spectrum, employers, and service providers. Journal of Vocational Rehabilitation.
   Web of Science ®Google Scholar
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. hdl:1942/28116. S2CID 15926286. https://doi.org/10.1177/001316446002000104
   Web of Science ®Google Scholar
• DeVault, D., Artstein, R., Benn, G., Dey, T., Fast, E., Gainer, A., Georgila, K., Gratch, J., Hartholt, A., Lhommet, M., Lucas, G., Marsella, S., Morbini, F., Nazarian, A., Scherer, S., Stratou, G., Suri, A., Traum, D., Wood, R., … Morency, L.-P. (2014). SimSensei kiosk: A virtual human interviewer for healthcare decision support. Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems (pp. 1061–1068).
   Google Scholar
• Ekman, P. (1993). Facial expression and emotion. The American Psychologist, 48(4), 384–392. https://doi.org/10.1037/0003-066X.48.4.384
   PubMed Web of Science ®Google Scholar
• Farzana, S., Valizadeh, M., & Parde, N. (2020). Modeling dialogue in conversational cognitive health screening interviews. Proceedings of the 12th Language Resources and Evaluation Conference (pp. 1167–1177).
   Google Scholar
• Frith, C. (2009). Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3453–3458. https://doi.org/10.1098/rstb.2009.0142
   PubMed Web of Science ®Google Scholar
• Galley, N., Betz, D., & Biniossek, C. (2015). Fixation durations-why are they so highly variable? Das ende von rational choice? Zur Leistungsf ̈ahigkeit der Rational-Choice-Theorie, 93. https://doi.org/10.13140/RG.2.1.3128.1769
   Google Scholar
• Gay, V., Leijdekkers, P., Agcanas, J., Wong, F., & Wu, Q. (2013). CaptureMyemotion: Helping autistic children understand their emotions using facial expression recognition and mobile technologies. Bled eConference.
   Google Scholar
• Giannakos, M. N., Sharma, K., Pappas, I. O., Kostakos, V., & Velloso, E. (2019). Multimodal data as a means to understand the learning experience. International Journal of Information Management, 48, 108–119. https://doi.org/10.1016/j.ijinfomgt.2019.02.003
   Web of Science ®Google Scholar
• Jurafsky, D., & Shriberg, E. (1997). Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual.
   Google Scholar
• Kenny, L., Hattersley, C., Molins, B., Buckley, C., Povey, C., & Pellicano, E. (2016). Which terms should be used to describe autism? Perspectives from the UK autism community. Autism, 20(4), 442–462. https://doi.org/10.1177/1362361315588200
   PubMed Web of Science ®Google Scholar
• Kim, H. G., Cheon, E. J., Bai, D. S., Lee, Y. H., & Koo, B. H. (2018). Stress and heart rate variability: A meta-analysis and review of the literature. Psychiatry Investigation, 15(3), 235–245. https://doi.org/10.30773/pi.2017.08.17
   PubMed Web of Science ®Google Scholar
• Maenner, M. J., Shaw, K. A., Bakian, A. V., Bilder, D. A., Durkin, M. S., Esler, A., Furnier, S. M., Hallas, L., Hall Lande, J., Hudson, A., Hughes, M. M., Patrick, M., Pierce, K., Poynter, J. N., Salinas, A., Shenouda, J., Vehorn, A., Warren, Z., Constantino, J. N., … Cogswell, M. E. (2021). Prevalence and characteristics of autism spectrum disorder among children aged 8 years — autism and developmental disabilities monitoring network, 11 sites, United States, 2018. MMWR Surveillance Summaries, 70(11), 1. https://doi.org/10.15585/mmwr.ss7011a1
   Web of Science ®Google Scholar
• Maras, K. L., Norris, J. E., Nicholson, J., Heasman, B., Remington, A., & Crane, L. (2020). Ameliorating the disadvantage for autistic job seekers: An initial evaluation of adapted employment interview questions. Autism, 25(4), 1060–1075. https://doi.org/10.1177/1362361320981319
   PubMed Web of Science ®Google Scholar
• Mayring, P. (2019). Qualitative content analysis: Demarcation, varieties, developments. Forum: Qualitative Social Research, 20, 3. https://doi.org/10.17169/fqs-20.3.3343
   Google Scholar
• McCarthy, J., & Goffin, R. (2004). Measuring job interview anxiety: Beyond weak knees and sweaty palms. Personnel Psychology, 57(3), 607–637. https://doi.org/10.1111/j.1744-6570.2004.00002.x
   Web of Science ®Google Scholar
• McGovern, T. V., & Tinsley, H. E. (1978). Interviewer evaluations of interviewee nonverbal behavior. Journal of Vocational Behavior, 13(2), 163–171. https://doi.org/10.1016/0001-8791(78)90041-6
   Web of Science ®Google Scholar
• Microsoft Azure Cognitive Services [Computer Software]. (2022). https://azure.microsoft.com/en-us/services/cognitive-services/
   Google Scholar
• Microsoft Azure Cosmos DB [Computer Software]. (2022). https://azure.microsoft.com/en-us/services/cosmos-db/
   Google Scholar
• Migovich, M., Korman, A., Wade, J., & Sarkar, N. (2021). Design and validation of a stress detection model for use with a VR based interview simulator for autistic young adults. International Conference on Human-Computer Interaction (pp. 580–588). https://doi.org/10.1007/978-3-030-78092-0_40
   Google Scholar
• Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.
   Google Scholar
• Nguyen, L. S., Frauendorfer, D., Mast, M. S., & Gatica-Perez, D. (2014). Hire me: Computational inference of hirability in employment interviews based on nonverbal behavior. IEEE Transactions on Multimedia, 16(4), 1018–1031. https://doi.org/10.1109/TMM.2014.2307169
   Web of Science ®Google Scholar
• Nicholas, D. B., Hodgetts, S., Zwaigenbaum, L., Smith, L. E., Shattuck, P., Parr, J. R., Conlon, O., Germani, T., Mitchell, W., Sacrey, L., & Stothers, M. E. (2017). Research needs and priorities for transition and employment in autism: Considerations reflected in a “special interest group” at the international meeting for autism research. Autism Research, 10(1), 15–24. https://doi.org/10.1002/aur.1683
   PubMed Web of Science ®Google Scholar
• Nicholas, D. B., & Klag, M. (2020). Critical reflections on employment among autistic adults. Autism in Adulthood, 2(4), 289–295. https://doi.org/10.1089/aut.2020.0006
   PubMedGoogle Scholar
• Nivo [Computer Software]. (2022). https://nivo.rocks/
   Google Scholar
• React-A JavaScript library for building user interfaces [Computer Software]. (2022). https://reactjs.org/
   Google Scholar
• React Bootstrap Table [Computer Software]. (2022). http://allenfang.github.io/react-bootstrap-table/
   Google Scholar
• React Chat Elements [Computer Software]. (2022). https://detaysoft.github.io/docs-react-chat-elements
   Google Scholar
• Roux, A. M., Rast, J. E., Anderson, K. A., & Shattuck, P. T. (2017). National autism indicators report: Developmental disability services and outcomes in adulthood. Life Course Outcomes Program, AJ Drexel Autism Institute, Drexel University.
   Google Scholar
• Sarrett, J. C. (2017). Interviews, disclosures, and misperceptions: autistic adults’ perspectives on employment related challenges. Disability Studies Quarterly, 37(2). https://doi.org/10.18061/DSQ.V37I2.5524
   Google Scholar
• Sasson, N. J., Faso, D. J., Nugent, J., Lovell, S., Kennedy, D. P., & Grossman, R. B. (2017). Neurotypical peers are less willing to interact with those with autism based on thin slice judgments. Scientific Reports, 7(1), 1–10. https://doi.org/10.1038/srep40700
   PubMedGoogle Scholar
• Senju, A., & Johnson, M. H. (2009). Atypical eye contact in autism: Models, mechanisms and development. Neuroscience & Biobehavioral Reviews, 33(8), 1204–1214. https://doi.org/10.1016/j.neubiorev.2009.06.001
   PubMed Web of Science ®Google Scholar
• Shellenbarger, S. (2013). Just look me in the eye already. The Wall Street Journal. Retrieved March 4, 2022, from https://www.wsj.com/articles/SB10001424127887324809804578511290822228174
   Google Scholar
• Silani, G., Bird, G., Brindley, R., Singer, T., Frith, C., & Frith, U. (2008). Levels of emotional awareness and autism: An fMRI study. Social Neuroscience, 3(2), 97–112. https://doi.org/10.1080/17470910701577020
   PubMed Web of Science ®Google Scholar
• Smith, M. J., Fleming, M. F., Wright, M. A., Losh, M., Humm, L. B., Olsen, D., & Bell, M. D. (2015). Brief report: Vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. Journal of Autism and Developmental Disorders, 45(10), 3364–3369. https://doi.org/10.1007/s10803-015-2470-1
   PubMed Web of Science ®Google Scholar
• Smith, M. J., Ginger, E. J., Wright, K., Wright, M. A., Taylor, J. L., Humm, L. B., Olsen, D. E., Bell, M. D., & Fleming, M. F. (2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(10), 2450–2463. https://doi.org/10.1007/s10803-014-2113-y
   PubMed Web of Science ®Google Scholar
• Smith, M. J., Pinto, R. M., Dawalt, L., Smith, J. D., Sherwood, K., Miles, R., Taylor, J., Hume, K., Dawkins, T., Baker-Ericzén, M., Frazier, T., Humm, L., & Steacy, C. (2020). Using community-engaged methods to adapt virtual reality job-interview training for transition-age youth on the autism spectrum. Research in Autism Spectrum Disorders, 71, 101498. https://doi.org/10.1016/j.rasd.2019.101498
   PubMed Web of Science ®Google Scholar
• Smith, M. J., Sherwood, K., Ross, B., Smith, J. D., DaWalt, L., Bishop, L., Humm, L., Elkins, J., & Steacy, C. (2021). Virtual interview training for autistic transition age youth: A randomized controlled feasibility and effectiveness trial. Autism, 25(6), 1536–1552. https://doi.org/10.1177/1362361321989928
   PubMed Web of Science ®Google Scholar
• Spain, D., Sin, J., Linder, K. B., McMahon, J., & Happé, F. (2018). Social anxiety in autism spectrum disorder: A systematic review. Research in Autism Spectrum Disorders, 52, 51–68. https://doi.org/10.1016/j.rasd.2018.04.007
   Web of Science ®Google Scholar
• Stolcke, A., Coccaro, N., Bates, R., Taylor, P., Van Ess-Dykema, C., Ries, K., Shriberg, E., Jurafsky, D., Martin, R., & Meteer, M. (2000). Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3), 339–373. https://doi.org/10.1162/089120100561737
   Web of Science ®Google Scholar
• Strickland, D. C., Coles, C. D., & Southern, L. B. (2013). JobTIPS: A transition to employment program for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(10), 2472–2483. https://doi.org/10.1007/s10803-013-1800-4
   PubMed Web of Science ®Google Scholar
• Sunburst Chart [Computer Software]. (2022). https://www.npmjs.com/package/sunburst-chart
   Google Scholar
• Taylor, J. L., DaWalt, L. S., Marvin, A. R., Law, J. K., & Lipkin, P. (2019). Sex differences in employment and supports for adults with autism spectrum disorder. Autism, 23(7), 1711–1719. https://doi.org/10.1177/1362361319827417
   PubMed Web of Science ®Google Scholar
• Trevisan, D. A., Roberts, N., Lin, C.Y. -H., Birmingham, E., & Hadjikhani, N. (2017). How do adults and teens with self-declared autism spectrum disorder experience eye contact? A qualitative analysis of first-hand accounts. Plos One, 12(11), 1–22. https://doi.org/10.1371/journal.pone.0188446
   Web of Science ®Google Scholar
• Unity 3D 2019.3 [Computer Software]. (2022). https://unity.com/releases/2019- 3
   Google Scholar
• Wang, G., Zhang, J., & Fu, G. -S. (2018). Predicting student behaviors and performance in online learning using decision tree. 2018 Seventh International Conference of Educational Innovation through Technology (EITT) (pp. 214–219). https://doi.org/10.1109/EITT.2018.00050
   Google Scholar
• Wehman, P., Taylor, J., Brooke, V., Avellone, L., Whittenburg, H., Ham, W., Brooke, A. M., & Carr, S. (2018). Toward competitive employment for persons with intellectual and developmental disabilities: What progress have we made and where do we need to go. Research and Practice for Persons with Severe Disabilities, 43(3), 131–144. https://doi.org/10.1177/1540796918777730
   Web of Science ®Google Scholar
• Whelpley, C. E., & May, C. P. (2022). Seeing is disliking: Evidence of bias against individuals with autism spectrum disorder in traditional dfjob interviews. Journal of Autism and Developmental Disorders, 1–12. https://doi.org/10.1007/s10803-022-05432-2
   Web of Science ®Google Scholar
• Xu, Q., Cheung, S. S., & Soares, N. (2015). LittleHelper: An augmented reality glass application to assist individuals with autism in job interview. 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) (pp. 1276–1279). https://doi.org/10.1109/APSIPA.2015.7415480
   Google Scholar