Examining perceptions and outcomes of AI versus human course assistant discussions in the online classroom

ABSTRACT

As technologies such as artificial intelligence (AI) and other forms of machine communication become more popular and readily available, the opportunities for use in an online class increase. This replication and extension sought to understand and test the use of AI versus human communication in an online learning space—specifically the learning management system (LMS) discussion board. More specifically, the differences in student perceptions and outcomes between those with a human course assistant and those assigned to an AI course assistant were examined. These perceptions were studied through the presence and absence of the identity heuristic and helper heuristic as outlined in previous literature. The study employed a four-condition randomized quasiexperimental design. Outcomes of interest included credibility, task attraction, social attraction, computer-mediated competence, student communication satisfaction, affect toward instructor, and affect toward taking classes with the instructor. Findings supported previous work regarding importance of helper heuristics and highlighted the preference for a human over AI/machine educational agent.

KEYWORDS:

• AI; online discussion boards; communication technology; instructional communication
• human–machine communication

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Although there is a large amount of research on robots, machines, and credibility from outside of communication, these tend to confound credibility and trust, and/or are not using similar concepts, making the literature from that specific area difficult to generalize from (see Schroeder et al., 2021).

2 With the goal of making replication and extensions of research easier (Bowman & Keene, 2018; Bowman & Spence, 2020; Bowman et al., 2023), the experimental materials for this study can be found at https://osf.io/hab9j.