Taking the Chat out of Chatbot? Collecting User Reviews with Chatbots and Web Forms

ABSTRACT

Firms are beginning to use chatbots to collect information from users (e.g., online reviews), a task traditionally done through forms. We used two experiments to investigate how collecting reviews using chatbots in place of forms might impact user experience and, in turn, alter the nature of the reviews submitted by users. Study 1 compares chatbots to web forms, finding that chatbots may improve perceived efficiency but otherwise have no discernible effect on satisfaction and usage intention. At the same time, chatbot usage results in shorter, lower-quality reviews. While structured templates enhance form-based reviews, introducing structure to chatbot interactions does not positively impact satisfaction and usage intent, potentially making the process less efficient. Nonetheless, a structured chatbot approach yields longer reviews and mitigates declines in quality. Adding structure to a chatbot takes the chat out of the chatbot, turning the interaction from casual conversation to a formal process, as demonstrated in Study 2. Hence, while this structured approach for chatbots improves review quality without harming satisfaction and usage intent, it may not be the most effective method for enhancing the reviewer experience. Our research shows that the chatbot and structure pulled users towards contradicting genre rules (the social structures that guide technology use) and triggered users to subconsciously enact distinctly different thought patterns as they composed reviews.

KEYWORDS:

• Chatbot
• AI
• form
• online review
• structure
• review characteristics
• genre rules

Disclosure statement

No potential conflicts of interest are reported by the authors(s).

Supplementary Materials

Supplemental data for this article can be accessed here.

Additional information

Notes on contributors

Agrim Sachdeva

Agrim Sachdeva ([email protected]) is a Ph.D. candidate in Information Systems at the Kelley School of Business, Indiana University. He will be joining the Eller College of Management at the University of Arizona in 2024. His research focuses on AI, especially conversational agents, and his work has appeared in Journal of Management Information Systems.

Antino Kim

Antino Kim ([email protected]) is an Associate Professor of Information Systems and Grant Thornton Scholar at the Kelley School of Business, Indiana University. He earned his Ph.D. in Information Systems from the Foster School of Business at the University of Washington. Dr. Kim’s research interests include: AI and human interaction; misinformation and social media; digital piracy; and markets for information goods. His research has appeared in Information Systems Research, Journal of Management Information Systems, Management Science, and MIS Quarterly, among other outlets.

Alan R. Dennis

Alan R. Dennis ([email protected]) is a Professor of Information Systems and holds the John T. Chambers Chair of Internet Systems in the Kelley School of Business at Indiana University. His research focuses on four main themes: AI agents; fake news on social media; team collaboration; and information security He is a past President, Fellow, and LEO awardee of the Association for Information Systems.