Multimodal job interview simulator for training of autistic individuals

ABSTRACT

Autistic individuals face difficulties in finding and maintaining employment, and studies have shown that the job interview is often a significant barrier to obtaining employment. Prior computer-based job interview training interventions for autistic individuals have been associated with better interview outcomes. These previous interventions, however, do not leverage the use of multimodal data that could give insight into the emotional underpinnings of autistic individuals’ challenges in job interviews. In this article, the authors present the design of a novel multimodal job interview training platform called CIRVR that simulates job interviews through spoken interaction and collects eye gaze, facial expressions, and physiological responses of the participants to understand their stress response and their affective state. Results from a feasibility study with 23 autistic participants who interacted with CIRVR are presented. In addition, qualitative feedback was gathered from stakeholders on visualizations of data on CIRVR’s visualization tool called the Dashboard. The data gathered indicate the potential of CIRVR along with the Dashboard to be used in the creation of individualized job interview training of autistic individuals.

KEYWORDS:

• autism
• employment
• job interview
• multimodal data
• virtual reality

Acknowledgements

The authors gratefully acknowledge the support provided by Microsoft AI for Accessibility Program and by the National Science Foundation under grant numbers 1936970 and 2033413 for this project. They would like to thank the reviewers and the Editor-in-Chief of Assistive Technology Journal for their constructive and insightful comments which helped clarify the contributions of the work and in turn, strengthen the manuscript. The authors are solely responsible for the content of this article. None of the authors have any financial conflict of interest.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

¹ Autistic individuals have shown preference for identity-first language (“autistic”) rather than person-first (“person with autism”) (Kenny et al., 2016). Thus, the authors have used this terminology throughout the article.

² https://www.dedoose.com/.

⁴ Fixation duration is the time duration of the eyes gazing upon an object in its surroundings, where a fixation lasts approximately 250 milliseconds, but may vary, either shorter or longer (Galley et al., 2015).

³ IRB Approval #191277.

⁵ Some sessions were conducted during the COVID-19 pandemic and the consent forms and post-session questionnaires were administered electronically and post-session scores were lost for 3 participants due to technical errors (e.g., corrupt file, unsaved file lost).

Additional information

Funding

The work was supported by the National Science Foundation [1936970, 2033413]; AI for Accessibility Grant from Microsoft [No number].