1,673
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

Artificial Intelligence in Influencer Marketing: A Mixed-Method Comparison of Human and Virtual Influencers on Instagram

&

Abstract

The prominence and profitability of influencer marketing have facilitated a proliferation of virtual influencers—fictitious digital personalities created and managed using artificial intelligence. Virtual influencers may offer advertisers greater creative control and yield greater engagement than human influencers. Hence, we investigated these claims by comparing the persuasion strategies and outcomes between human and virtual influencers. We retrieved 99,680 English-language Instagram posts uploaded by 424 human and virtual influencers within the beauty, fashion, and lifestyle domains from 2020 to 2022. Dictionary-based sentiment analysis (replicated across the AFINN and Bing lexicons) indicated that Instagram posts from both types of influencers predominantly conveyed positive sentiment. Latent Dirichlet allocation topic modeling revealed that both types of influencers asserted opinion leadership differently: Human influencers engaged in active self-promotion, while virtual influencers emphasized their identity. A natural experiment found that human influencers elicited greater engagement than virtual influencers. Influencer tier also significantly interacted with Instagram verification to affect engagement. Theoretical contributions, managerial implications, and directions for future research are discussed.

Technological advancements in automation and artificial intelligence (AI) have consistently evoked public concern regarding job displacement (Hiort Citation2022). While such sentiments often pertain to low-wage workers in manufacturing and service industries, job losses engendered by AI may also occur in influencer marketing (Hiort Citation2022; Ong Citation2020). Notably, the substantial advertising expenditure on influencer marketing has facilitated the proliferation of virtual influencers (VIs): fictitious digital personalities created and managed using AI (Arsenyan and Mirowska Citation2021; Moustakas et al. Citation2020).

In recent years, VIs have obtained greater engagement or advertising effectiveness (Wang Citation2006) than human influencers (HIs) and thus have become the preferred spokespersons for advertising campaigns (Moustakas et al. Citation2020). Advertisers have also touted VIs as less-risky alternatives than HIs, who may commit transgressions and incur negative spillover effects on their endorsements (Bradley Citation2021). In contrast, such reputational risks are less applicable to VIs, which are robots that offer advertisers unfettered control over their social media persona and content (Bradley Citation2021; Moustakas et al. Citation2020; Ong Citation2020). Since VIs are not constrained by geographic and physical boundaries, they are also constantly available to endorse advertising campaigns (Bradley Citation2021).

Despite VIs’ rising popularity, limited research has empirically assessed their persuasion strategies and outcomes (Moustakas et al. Citation2020; Stein et al. Citation2022; Thomas and Fowler Citation2021). Exploratory research utilized case studies to examine @lilmiquela—a VI specializing in beauty, fashion, and lifestyle with millions of followers—and the ethical considerations surrounding VIs’ inauthenticity (Lou et al. Citation2023). Moustakas et al. (Citation2020) also conducted semi-structured interviews to understand experts’ perceived effectiveness of VIs.

Moreover, recent research conducted experiments to assess VIs’ persuasiveness. For instance, Thomas and Fowler (Citation2021) examined a fictitious VI. Meanwhile, a study by Stein et al. (Citation2022) featured @CodeMiko—an existing VI with 917,000 followers that live-streams its gaming experiences on Twitch. Advertising scholars have also increasingly juxtaposed existing and established VIs (e.g., @lilmiquela) and HIs with many followers (Franke, Groeppel-Klein, and Müller Citation2022; Lee and Ham Citation2023; Yang et al. Citation2023). In particular, these studies compared how HIs and VIs affected consumers’ attitudes toward the endorsement and purchase intention (Franke, Groeppel-Klein, and Müller Citation2022; Lee and Ham Citation2023; Yang et al. Citation2023). However, few studies have assessed the growing community of VIs in nascent phases of establishing opinion leadership.

Apart from these studies (e.g., Arsenyan and Mirowska Citation2021; Stein et al. Citation2022; Thomas and Fowler Citation2021), limited research has juxtaposed consumers’ parasocial interaction (PSI) with HIs and VIs. PSI refers to the illusionary and one-sided psychological connections (e.g., perceived friendship, relational intimacy) that consumers experience during episodic exposures to media personalities (Dibble, Hartmann, and Rosaen Citation2016; Kim Citation2022; Stein et al. Citation2022). Since the persuasiveness of influencer endorsements is attributed to PSI (Gong and Li Citation2017; Kim Citation2022), we propose a comparative study to empirically assess whether VIs outperform HIs in this regard.

This study addresses the aforementioned research gaps with a mixed-method approach. Upon retrieving Instagram data regarding HIs and VIs within the beauty, fashion, and lifestyle domains from 2020 to 2022, study 1 utilizes computational methods to compare their persuasion strategies. In this context, persuasion strategies refer to how HIs and VIs establish opinion leadership and achieve their intended persuasion outcomes. These strategies include the emotional appeal and topics discussed in their Instagram posts. To this end, study 1 assesses the emotional valence and intensity captured in, as well as the key themes of, Instagram posts uploaded by HIs and VIs. Using the same Instagram dataset, study 2 conducts a natural experiment to analyze how influencer type, influencer tier, and Instagram verification determine their persuasiveness. This is operationalized using engagement—a metric of advertising effectiveness that is calculated by aggregating the number of likes and comments obtained for each Instagram post, then dividing it by the influencer’s follower count at the time of posting (Chacon Citation2018; Corporate Finance Institute Citation2022). To this end, study 2 juxtaposes the uncanny valley hypothesis and computers as social actors (CASA) framework as competing paradigms to determine whether VIs obtain greater engagement than HIs. Study 2 also evaluates how influencer tier and Instagram verification moderate engagement.

Our findings may provide several implications. Conceptually, these studies clarify scholarly understanding about VIs by extending human–AI interaction to the topic of influencer marketing. Study 1 illuminates whether VIs’ Instagram posts emulate or differ from the emotional valence, emotional intensity, and topics conveyed in HIs’ Instagram posts. Study 2 advances influencer marketing research by investigating whether the CASA paradigm prevails over the uncanny valley hypothesis in accounting for VIs’ persuasiveness relative to HIs. Study 2 also contributes to influencer marketing literature by assessing how influencer tier and Instagram verification shape persuasion. Particularly, study 2’s nuanced distinction of influencer tier improves on the binary classification of influencer tier (low vs. high) in existing research. Moreover, study 2 uncovers how Instagram verification denotes influencers’ opinion leadership, thereby shaping their persuasiveness. Practically, these studies will inform advertisers about whether VIs are effective alternatives to HIs. Observations regarding influencer tier and Instagram verification may also assist advertisers in selecting appropriate Instagram influencers as campaign spokespersons.

HIs

HIs are laypersons that have garnered clout on social media by amassing many followers (De Veirman, Cauberghe, and Hudders Citation2017; Lou and Yuan Citation2019). In establishing their identity as “authentic personal brands,” HIs foster and sustain PSI with their followers by disclosing and documenting their daily lives (Lee et al. Citation2022; Lou Citation2022). HIs also assert expertise in niche communities (e.g., beauty, fashion, food, and travel) by publicizing original content including product recommendations and reviews (De Veirman, Cauberghe, and Hudders Citation2017; Lee and Eastin Citation2021). Through integrating their personal narratives with sponsored content (Lee et al. Citation2022), HIs function as opinion leaders or “trusted tastemakers” who enhance brand awareness, brand attitudes, product evaluations, and purchase intentions (De Veirman, Cauberghe, and Hudders Citation2017; Lou and Yuan Citation2019).

HIs’ endorsements are regarded as credible electronic word-of-mouth that provides genuine brand and product information (Lee et al. Citation2022; Lou and Yuan Citation2019). As such, HIs’ endorsements transcend consumers’ growing aversion to celebrity endorsements and brand-generated advertisements (Lee and Eastin Citation2021; Lou, Tan, and Chen Citation2019). Considering the persuasiveness and popularity of HIs’ endorsements, brands substantially increased their advertising expenditure by engaging HIs as brand ambassadors or product endorsers (Ong Citation2020).

HIs and Persuasion

Alongside the exponential growth of influencer marketing, scholars have analyzed how credibility perceptions of HIs affect persuasion. Specifically, consumers’ perceived expertise and trustworthiness of HIs affected brand awareness, which influenced purchase intentions (Gong and Li Citation2017; Lou and Yuan Citation2019). Consumers’ perceived homophily and attractiveness of HIs also shaped their trust in sponsored content, which impacted brand awareness and purchase intentions (Lou and Yuan Citation2019). Apart from these subdimensions, consumers’ overarching credibility perceptions of HIs enhanced perceived advertising effectiveness, which improved attitudes toward the endorsement and featured brand (Lee and Ham Citation2023). Credibility perceptions of HIs also heightened consumers’ intention to partake in a brand’s corporate social responsibility engagement, thus improving the brand’s reputation (Yang et al. Citation2023).

In addition to credibility perceptions, consumers’ perceived authenticity of HIs enhanced advertising effectiveness (Park et al. Citation2021). Specifically, consumers’ perceived sincerity and transparency improved their attitudes toward the HI, intention to follow the HI, and purchase intentions (Lee and Eastin Citation2021).

Past studies also evaluated consumers’ PSI with HIs, which is a precursor of attitudes and behavioral intentions (Gong and Li Citation2017; Kim Citation2022). PSI conventionally pertained to consumers’ imagined friendship and relational intimacy with media personalities, including celebrities and news broadcasters (Horton and Wohl Citation1956; Rubin, Perse, and Powell Citation1985). However, in the prevailing media environment, consumers can engage in conversations with HIs by liking and commenting on HIs’ social media content (Brooks, Drenten, and Piskorski Citation2021). This results in a “trans-parasocial” relationship where consumers can establish reciprocal, co-constructed, and intimate relationships with HIs (Kim Citation2022; Kim and Song Citation2016; Lou Citation2022). Despite this, consumers’ efforts often “go unnoticed amid the vast online audience” (Stein et al. Citation2022, 4). Therefore, consumers’ PSI with HIs refer to their imagined and one-sided psychological connections that are reminiscent of fan letters to celebrities (Stein et al. Citation2022).

Scholars also conceptualized PSI as “an illusory user experience” that is “confined to the media exposure situation” (Dibble, Hartmann, and Rosaen Citation2016, 23; Hartmann and Goldhoorn Citation2011; Horton and Wohl Citation1956). This is antecedent to and distinct from parasocial relationships, which are enduring emotional bonds that consumers develop upon successive exposures to media personalities (Dibble, Hartmann, and Rosaen Citation2016; Giles Citation2002). Beyond examining consumers’ psychological connections with media personalities, recent research (Dibble, Hartmann, and Rosaen Citation2016; Stein et al. Citation2022) explicated PSI into consumers’ cognitive evaluations (e.g., close observations of the media personality), affective responses (e.g., liking), and behavioral intentions (e.g., sharing their opinion with the media personality). Guided by Stein et al.’s (Citation2022) operationalization, this study measured engagement, which reflects the affective (number of likes) and behavioral (number of comments) dimensions of PSI.

Overall, advertising research extensively examined how HIs’ endorsements engendered varying persuasion and reconceptualized PSI in the context of influencer marketing. However, with the exception of Stein et al. (Citation2022), no studies assessed its generalizability to VIs. Therefore, the following section synthesizes differences between HIs and VIs, as well as its potential implications on persuasion.

VIs

Like HIs, VIs inform consumers’ attitudes and behaviors by serving as brand ambassadors and providing product endorsements (Bradley Citation2021; Moustakas et al. Citation2020; Thomas and Fowler Citation2021). Despite fulfilling similar roles, VIs possess several noteworthy differences from HIs.

First, HIs are represented by themselves or through management agencies. Meanwhile, VIs are ageless and fictitious social media personalities created and managed using AI (Arsenyan and Mirowska Citation2021; Moustakas et al. Citation2020). Before creating VIs, AI experts use social listening tools to identify their target audience’s demographics and psychographics. Then, these observations are incorporated into the VI’s appearance using computer graphics (Choudhry et al. Citation2022; Stein et al. Citation2022). These findings also inform the VI’s persona and backstory, which are generated by scriptwriters (Bradley Citation2021).

After creating the VI, minimal human assistance is required to manage the VI’s social media activities (Hiort Citation2022). For example, computer-generated imagery is utilized to embed the VI into existing photographs or videos (Bradley Citation2021). This results in mixed-reality social media content wherein VIs are interacting with HIs in the physical environment (Ham et al. Citation2023). Using unsupervised machine learning (ML), natural language processing (NLP) is conducted to systematically analyze the linguistic structure of consumers’ social media posts regarding specific brands and products (Kietzmann, Paschen, and Treen Citation2018; Thomas and Fowler Citation2021). Then, the unlabeled dataset of consumers’ social media posts will be clustered to identify key patterns in consumer behavior (Thomas and Fowler Citation2021). Moreover, unsupervised ML is utilized to generate VIs’ Instagram posts that are congruent with the brand’s image (Thomas and Fowler Citation2021). Unsupervised ML also enables VIs to iteratively learn from consumers’ social media posts (Kietzmann, Paschen, and Treen Citation2018), which continually enhances VIs’ simulation of human speech and informs their response to consumers’ comments (Thomas and Fowler Citation2021). Altogether, VIs are “sentient” avatars that utilize unsupervised ML to incorporate consumers’ preferences when creating content (Moustakas et al. Citation2020; Thomas and Fowler Citation2021) and managing consumer relationships. In doing so, VIs leverage unsupervised ML to facilitate psychological connections or PSI with consumers (Lou Citation2022).

Second, VIs offer advertisers greater creative control than HIs (Moustakas et al. Citation2020). In comparison, HIs possess greater autonomy in determining their social media persona and content (Moustakas et al. Citation2020; Thomas and Fowler Citation2021). Additionally, HIs have greater personal freedom than VIs in shaping PSI with consumers. That is, HIs can decide on the frequency and intimacy of self-disclosure, which are key determinants in establishing and strengthening profound relational bonds with consumers (Kim and Song Citation2016).

While HIs have a personal life that is beyond advertisers’ control, VIs exist solely on social media to endorse brands and products (Thomas and Fowler Citation2021). Since VIs do not possess a physical presence in reality, they are analogous to digital extensions of mannequins in brick-and-mortar retail outlets (Stein et al. Citation2022). VIs also possess conversational capabilities that surpass the transactional functions of online chatbots, enabling them to transcend geographic and physical limitations to continually interact with their followers (Stein et al. Citation2022).

Overall, HIs and VIs are opinion leaders on social media. Yet, they possess varying autonomy in determining their social media persona and activities (Moustakas et al. Citation2020; Thomas and Fowler Citation2021).

VIs and Persuasion

HIs and VIs also persuade consumers differently: Past studies have identified HIs’ credibility and authenticity as determinants of their persuasiveness (Gong and Li Citation2017; Lou and Yuan Citation2019; Park et al. Citation2021). However, VIs lacked authenticity and credibility (Arsenyan and Mirowska Citation2021; Lou et al. Citation2023; Moustakas et al. Citation2020). Instead, recent research attributed VIs’ persuasiveness to their technological novelty and the entertainment value from its creative storytelling (Choudhry et al. Citation2022; Lou et al. Citation2023; Stein et al. Citation2022). Furthermore, consumers responded more favorably when VIs acknowledged their artificiality and advocated similar social values (Lou et al. Citation2023). Thus, HIs’ persuasiveness rests upon source characteristics, while VIs’ persuasiveness is contingent upon their content and technological attributes.

Despite this, HIs and VIs are capable of facilitating PSI with consumers, which enhanced their persuasiveness (Kim and Song Citation2016; Lou Citation2022). HIs foster consumers’ perceived friendships and psychological connections by providing self-disclosure about their life experiences and creating content tailored to consumers’ requests (Gong and Li Citation2017; Lou Citation2022). Meanwhile, VIs cultivated PSI by responding frequently to consumers (Choudhry et al. Citation2022). Since consumers were intrigued with VIs, they expressed greater enjoyment when interacting with VIs (relative to HIs) and indicated greater intention to interact with VIs in the future (Stein et al. Citation2022). As a result, consumers were more likely to like, comment, and share VIs’ social media content than that of HIs (Choudhry et al. Citation2022).

Considering these differences, it is insufficient to extrapolate observations from HIs to inform scholarly understanding of VIs. Hence, further investigation is needed to assess whether HIs and VIs utilize varying strategies to facilitate persuasion.

Study 1: Persuasion Strategies of Human and Virtual Influencers

Recent research has increasingly compared the source characteristics (e.g., perceived authenticity and humanlikeness) of HIs and VIs, as well as their implications on persuasion (Stein et al. Citation2022; Thomas and Fowler Citation2021; Yang et al. Citation2023). Yet, this theoretical emphasis overlooks the roles of content attributes in influencer marketing. According to the social media influencer value (SMIV) model, consumers displayed greater trust and purchase intention when they perceived the HI’s endorsement as informative and entertaining (Lou and Yuan Citation2019). Moreover, consumers perceived the HI’s public service announcements (PSAs) more favorably and exhibited greater likelihood of adopting preventive health behaviors when the PSA utilized plural personal pronouns and emphasized societal-level benefits (Looi et al. Citation2022). Notably, the importance of content attributes was replicated among VIs. Consumers overcame their initial fear and revulsion when the VI’s content was perceived as creative and entertaining (Choudhry et al. Citation2022; Lou et al. Citation2023).

To address the scarcity of research examining content attributes in influencer marketing, study 1 compares the persuasion strategies employed by HIs and VIs. Specifically, study 1 systematically analyzes whether HIs and VIs express emotions differently through their Instagram posts. Study 1 also investigates whether the Instagram posts from HIs and VIs focus on different topics.

VIs’ social media posts can be generated using unsupervised ML (Thomas and Fowler Citation2021), which reflects and reinforces the most frequently occurring content posted by consumers or influencers (Thomas and Fowler Citation2021). Depending on how these unsupervised ML algorithms are developed, VIs’ social media content may also reflect the emotional valence, emotional intensity, and topics referenced by successful brand-generated advertisements drafted by professional copywriters (Kietzmann, Paschen, and Treen Citation2018). Therefore, to establish their opinion leadership on Instagram, VIs may emulate “tried-and-tested” human-generated Instagram posts to elicit predictable and favorable outcomes (Thomas and Fowler Citation2021).

Comparatively, HIs often rely on their own discretion in creating original content (Lee and Eastin Citation2021). Although HIs may incorporate inputs from their management agencies and advertisers, they ultimately possess autonomy as digital content creators (De Veirman, Cauberghe, and Hudders Citation2017). As such, HIs may adhere to the most frequently occurring persuasion strategies in prevailing influencer campaigns and draw upon their personal successes from previous endorsements to inform content creation. Relative to those of VIs, HIs’ social media posts may feature different emotional valence, emotional intensity, and topics that engender less predictable outcomes. Therefore, this study explores:

RQ1: To what extent do the Instagram posts from VIs and HIs convey the same emotions?

RQ2: To what extent do the Instagram posts from VIs and HIs convey the same topics?

Method

Data Collection

To address the research questions, study 1 adhered to the following procedure: First, we identified 77 prominent VIs within the beauty, fashion, and lifestyle domains from VirtualHumans.org—an online repository documenting developments in VI marketing. Based on CrowdTangle’s preexisting lists of beauty, fashion, and lifestyle influencers, we identified 347 HIs with comparable follower count (relative to the VIs identified earlier). Despite the unequal sample sizes, this approach enhanced the comparability of HIs and VIs by minimizing differences in follower count. To mitigate confounding factors, this study also ensured that the HIs matched specific VIs with similar demographics (age, gender, ethnicity) and Instagram verification. Using CrowdTangle, we retrieved 99,680 English-language Instagram posts uploaded by 424 HIs and VIs from January 1, 2020, to January 1, 2022. To prevent sampling biases, we randomly selected 10,000 Instagram posts (nhuman = 5,000 and nvirtual = 5,000) for further analyses.

Data Analysis

To address RQ1, we conducted automated, dictionary-based sentiment analysis using RStudio to analyze the emotions in HIs’ and VIs’ persuasion strategies on Instagram. Based on scholarly recommendations (Demirel Citation2020, Citation2022; Guo et al. Citation2016; Hayes et al. Citation2020; Liu, Burns, and Hou Citation2017; Yun et al. Citation2020), sentiment analysis is appropriate as it quantifies the sentiment and polarity of words (Guo et al. Citation2016), which indicates the emotional valence and intensity respectively. This approach also enabled systematic textual analysis of the linguistic structure in each Instagram post and the entire corpus of Instagram posts.

Sentiment analysis comprises dictionary-based, ML, and deep-learning approaches. Dictionary-based approaches utilize a compilation of English-language opinion words (i.e., lexicon) to compute the emotional tone of textual data based on the presence or frequency of words conveying positive or negative emotions (Chauhan et al. Citation2023; Hayes et al. Citation2020). Machine-learning (ML) approaches utilize manually coded textual datasets to train AI algorithms for predicting the sentiment of other large and unlabeled textual data (Brandwatch Citation2020). Meanwhile, deep learning is regarded as hierarchical ML that utilizes artificial neural networks (i.e., complex information processing units that mimic the structure of a biological brain) to interpret the definition and underlying context of textual datasets (Al-Qablan et al. Citation2023; Zhang, Wang, and Liu Citation2018).

The performance of these approaches has varied across social media platforms and its intended uses. While dictionary-based approaches “deliver outstanding outcomes across several application disciplines” (Chauhan et al. Citation2023, 4), past studies noted that ML and deep-learning approaches performed better when analyzing the sentiment of tweets (Al-Qablan et al. Citation2023; Chauhan et al. Citation2023). Meanwhile, dictionary-based and ML approaches performed similarly when categorizing words from Facebook posts into positive or negative sentiment (Dhaoui, Webster, and Tan Citation2017). Notably, Hayes et al. (Citation2020) observed that Brandwatch (n.d.)—which is a social media listening platform that utilizes automated and manual NLP—was “woefully unreliable” in detecting “post and brand sentiment polarity, specific emotions, and brand outcomes” (81). In fact, the dictionary-based approach yielded higher reliability with human coding than Brandwatch’s ML algorithm when categorizing the sentiment polarity of Instagram posts (see Hayes et al. Citation2020, 86). Considering this study’s contextual focus on Instagram posts, we adhered to Demirel’s (Citation2020, Citation2022) method by conducting dictionary-based sentiment analysis using Silge and Robinson’s (Citation2016) tidytext package from RStudio.

Dictionary-based sentiment analysis is also appropriate for this study’s context since it has been commonly utilized and validated in marketing research (Downer, Wells, and Crichton Citation2019; Demirel Citation2020, 2922). Moreover, Silge and Robinson’s (Citation2016) tidytext package provides access to several English lexicons, including AFINN and Bing. Since the methodological criticisms of dictionary-based approaches are directed to studies utilizing one lexicon, this study sought to mitigate the aforementioned methodological limitation and enhance the accuracy of sentiment scoring by replicating the findings across AFINN and Bing lexicons.

Prior to sentiment analysis, we tokenized the data by parsing data from sentences or phrases into words (Demirel Citation2020, Citation2022). Then, we removed stop words (i.e., words with semantically inconsequential meaning) from the tokenized data. The stop words in this study comprised a preexisting list from Silge and Robinson’s (Citation2016) tidytext package and a custom list of HTML characters (e.g., “amp”), web addresses (e.g., “https,” “t.co”), and punctuation. Specifically, the preexisting list of stop words is formulated using the onix (Lewis et al. Citation2004), SMART (Yencken Citation2011), and Snowball manuals (Snowball n.d.). It includes articles (e.g., “a,” “an,” “the”), personal pronouns (e.g., “my,” “our,” “his, “hers,” “their”), and auxiliary verbs (e.g., “am,” “is,” “are,” “was”).

We conducted separate sentiment analyses for each dataset (nhuman = 5,000 and nvirtual = 5,000) using the AFINN lexicon, which constitutes 2,477 English words scored from −5 to +5. The sign and magnitude indicate the emotional valence and intensity of each word in the tokenized dataset (i.e., word-level sentiment). Congruent with past studies (e.g., Puschmann and Powell Citation2018; Tian, Lai, and Moore Citation2018), words that did not belong to the AFINN lexicon were scored as zero, indicating neutral sentiment. Thereafter, the word-level sentiment was aggregated to derive the sentiment for each Instagram post (i.e., post-level sentiment). Instagram posts that obtained post-level sentiment >0 were classified as positive, while those with post-level sentiment <0 were categorized as negative. Meanwhile, Instagram posts that obtained post-level sentiment = 0 were regarded as neutral.

To enhance the accuracy of sentiment scoring, we replicated the findings with the Bing lexicon, which comprises 6,788 words and categorizes words in a binary manner (positive vs. negative). Similarly, words that did not belong to the Bing lexicon were classified as neutral. The overarching sentiment was then determined by the proportion of words conveying positive (vs. negative) sentiment. Finally, we utilized chi-square tests of independence to compare the overarching sentiment of Instagram posts from HIs and VIs.

For RQ2, we conducted unsupervised latent Dirichlet allocation (LDA) topic modeling, which is commonly utilized in computational social science to synthesize and classify large, unstructured textual data into key topics (Puschmann and Powell Citation2018; Tian, Lai, and Moore Citation2018; Maier et al. Citation2018). Adhering to recommendations from Guo et al. (Citation2016), this ML approach is well suited for discovering issues within HIs’ and VIs’ Instagram posts. Congruent with past studies (e.g., Maier et al. Citation2018; O’Halloran et al. Citation2017), the appropriate number of topics was determined using the following metrics: Perplexity refers to the topic model’s predictive likelihood. Semantic coherence refers to the quality and interpretability of topic models. Residuals pertain to the topic model’s goodness-of-fit. Using the searchK() function from the stm package in RStudio (Roberts, Stewart, and Tingley Citation2019), we ran tests from 5 to 10 topics and determined the optimal number of topics that had the highest perplexity and semantic coherence scores, while possessing the lowest residuals (khuman = 7, kvirtual = 8). Then, descriptions for each topic were generated using the stm package (Roberts, Stewart, and Tingley Citation2019).

Results

With regard to RQ1, the Instagram posts of HIs and VIs leaned toward positive sentiment, with some distinctions. Sentiment analysis using the AFINN lexicon indicated that VIs’ Instagram posts conveyed an overarching positive sentiment (median = 121, SD = 930.2) and contained more words with positive sentiment (n = 4,897; 97.94%) than with negative sentiment (n = 54; 1.08%). Additionally, there was a greater emotional intensity of words with positive sentiment (+1 to +3411) than with negative sentiment (−1 to −4). This positivity bias was corroborated with the Bing lexicon, which indicated a higher occurrence of words with positive sentiment (e.g., “love,” “beautiful,” “happy,” “amazing,” and “creative”) than with negative sentiment (e.g., “miss,” “crazy,” “apocalypse,” “bad,” and “dark”).

Similarly, sentiment analysis using the AFINN lexicon indicated that HIs’ Instagram posts conveyed an overarching positive sentiment (median = 64, SD = 89.66) and included a greater proportion of words with positive sentiment (n = 4,791; 95.82%) than with negative sentiment (n = 127; 2.54%). As indicated by the sentiment scores, the words with positive sentiment (+1 to +585) were more intense than the words with negative sentiment (−1 to −10). The positivity bias was replicated with the Bing lexicon, which indicated a higher occurrence of words with positive sentiment (e.g., “love,” “happy,” “favorite,” “perfect,” and “beautiful”) than negative sentiment (e.g., “miss,” “hard,” “bad,” “dark,” and “crazy”; see for exemplar Instagram posts).

Table 1. Exemplar Instagram post captions.

A chi-square test of independence revealed that the relation between influencer type (HIs vs. VIs) and sentiment of Instagram posts was significant, χ2(1, N = 9,861) = 7.46, p < .01. VIs were more likely to incorporate words with positive sentiment in their Instagram posts than were HIs ().

Figure 1. Sentiment analysis of Instagram posts (Bing lexicon).

Figure 1. Sentiment analysis of Instagram posts (Bing lexicon).

In addressing RQ2, LDA topic modeling revealed eight key themes regarding VIs’ identity, opinion leadership, and expertise. Specifically, VIs emphasized their identity (topics 3, 4, and 6) as a “virtual human” created using “cgi” and “digital art.” VIs also portrayed themselves as agents of positive social change (topic 7). Additionally, VIs asserted opinion leadership by emphasizing their status as a “virtual influencer,” “digital model,” or “cgi model” (topics 3, 4, and 5) that posted “new” and trending content (topic 1). While some themes pertained to VIs’ expertise in providing fashion inspiration (topics 2, 3, 5, 6, and 8), other themes highlighted VIs’ endorsement of beauty products (topics 4 and 6).

Comparatively, LDA topic modeling revealed seven key themes regarding HIs’ impression management, opinion leadership, and expertise. HIs engaged in positive self-presentation by expressing gratitude toward their personal experiences (topics 5 and 6). HIs also asserted opinion leadership by actively promoting their social media profiles, latest content uploads, and affiliate links to their endorsements (topics 1 and 3). While several themes pertained to HIs’ recommendations for beauty products (topics 2 and 7), others were related to fashion (topic 4). See for exemplar Instagram posts.

Table 2. Exemplar Instagram post captions from the topic models.

Altogether, VIs were more likely to incorporate words conveying positive emotion in their Instagram posts than were HIs (). HIs and VIs also asserted opinion leadership differently. Hence, in addition to their disparities in humanlikeness, the findings illuminate content differences.

Figure 2. Unsupervised latent Dirichlet allocation (LDA) topic models of Instagram posts.

Figure 2. Unsupervised latent Dirichlet allocation (LDA) topic models of Instagram posts.

Study 2: Persuasive Impact of HIs and VIs

Since extant literature juxtaposed prominent HIs and VIs with substantial following and Instagram verification (e.g., Arsenyan and Mirowska Citation2021; Lou et al. Citation2023; Stein et al. Citation2022; Thomas and Fowler Citation2021), study 2 addresses this research gap with an in-depth comparison of HIs and VIs spanning various stages of opinion leadership, including those with fewer followers and no Instagram verification. Specifically, study 2 examined how influencer type, tier, and Instagram verification affect persuasion outcomes.

Advertisers and researchers have disagreed on VIs’ persuasiveness. Advocates noted that advertising campaigns featuring VIs received greater engagement due to technological novelty (Bradley Citation2021; Lou et al. Citation2023). Other supporters contended that VIs would not engage in insensitive or socially irresponsible behaviors, unlike HIs, which would jeopardize their brand and product endorsements (Arsenyan and Mirowska Citation2021; Bradley Citation2021). However, critics argued that VIs’ inauthenticity and lack of transparency may trigger backlash from consumers (Thomas and Fowler Citation2021). As such, the luster of VIs may diminish once the technological novelty wears off. Detractors also underscored that VIs are not immune to controversy. In fact, VIs may commit transgressions if they are created using biased ML algorithms (Thomas and Fowler Citation2021). Given the contradictory evidence, further research is warranted to empirically assess VIs’ persuasiveness.

Engagement

Engagement—which is calculated in terms of the number of likes and comments elicited by each social media post—has been regarded by scholars (e.g., Wang Citation2006) as an indicator of advertising effectiveness. Similarly, advertisers utilized engagement to measure consumers’ level of interaction with social media content (Chacon Citation2018; Corporate Finance Institute Citation2022). Notably, engagement may also reflect consumers’ PSI with HIs and VIs. Congruent with Giles’ (Citation2002, 289) conceptualization of PSI, engagement is “a user response to a figure” as if they were a “personal acquaintance.” Engagement is also aligned with the operationalization of PSI in experimental research. For instance, advertising scholars (Kim Citation2022; Stein et al. Citation2022) adapted Schramm and Hartmann’s (Citation2008) PSI process scale and utilized self-reports to measure participants’ cognitive evaluations (e.g., attention to the persona), affective attitudes (i.e., empathy and liking of persona), and behavioral intentions (e.g., speaking out to the persona). As such, the number of likes and comments in engagement reflect the affective and behavioral dimensions of PSI respectively. Since participants often underestimate or overestimate their social media usage (Burnell et al. Citation2021), engagement also provides an accurate and ecologically valid assessment of consumer behavior relative to self-reports.

Uncanny Valley Hypothesis vs. CASA

This study draws upon the uncanny valley hypothesis and CASA as competing frameworks to determine whether VIs outperform HIs in eliciting engagement on Instagram. The uncanny valley hypothesis was introduced by Mori, MacDorman, and Kageki (Citation2012), whereby individuals display increasing affinity and acceptance for robots that are more humanlike. However, shortly before robots exceed the threshold for an ideal level of humanlikeness, individuals respond with fear and disgust, drastically decreasing their affinity and acceptance toward robots (MacDorman et al. Citation2009; Mori, MacDorman, and Kageki Citation2012; Rosenthal-Von Der Pütten and Krämer Citation2014). Therefore, the uncanny valley hypothesis predicts that humanlike robots will evoke negative affective responses (Mori, MacDorman, and Kageki Citation2012).

Recent research has criticized the uncanny valley hypothesis for being overly simplistic and its limited predictive validity in the prevailing media environment (Rosenthal-Von Der Pütten and Krämer Citation2014). However, a meta-analysis by Kätsyri et al. (Citation2015) noted that uncanny valley effects only arose when individuals perceived a mismatch between artificial and humanlike features (e.g., embedding a pair of artificial-looking eyes on a humanlike face). Uncanny valley effects were also observed among consumers’ interaction with VIs. Specifically, consumers perceived VIs as “eerie” and expressed initial difficulties in forming close psychological connections with VIs (Arsenyan and Mirowska Citation2021; Lou et al. Citation2023). Notably, Arsenyan and Mirowska (Citation2021) observed that consumers responded to VIs with emojis that were significantly less positive than HIs. Therefore, consumers’ negative affective responses toward VIs may manifest in terms of their engagement. In this study, uncanny valley effects may arise if consumers regard the artificiality of VIs as inconsistent with their physical surroundings in the Instagram posts (i.e., mixed reality). Consumers will then react to VIs’ Instagram posts with negative affect, decreased affinity, and lowered acceptance, reducing their intention to engage (i.e., like, comment, and share) with the Instagram posts from VIs than from HIs.

Conversely, consumers may prefer VIs over HIs in certain circumstances. The CASA framework originally suggested that individuals interacted with technologies like they did with real people (Gambino, Fox, and Ratan Citation2020). Although individuals were aware that technologies had no feelings, they mindlessly adhered to social conventions (e.g., politeness, reciprocity) and expectations (e.g., gender stereotypes) in human–computer interactions. However, recent theoretical extensions noted that individuals developed specific mental modes and scripts for human–computer interactions (Gambino, Fox, and Ratan Citation2020). Specifically, individuals may rely on technological affordances as judgmental heuristics to inform credibility perceptions of information from computer agents (Sundar Citation2008). In this study, consumers may utilize machine heuristics to assess VIs’ credibility. Since VIs’ content is generated using AI (Thomas and Fowler Citation2021), individuals may perceive VIs as more objective and more competent in performing complex and precise tasks than are HIs. Considering the uncanny valley hypothesis and CASA’s conflicting predictions, this study asks:

RQ3: Will the Instagram posts from VIs elicit significantly greater engagement than those from HIs?

Persuasive Impact of Influencer Characteristics

The persuasiveness of influencer endorsements is also contingent upon influencer tier and Instagram verification. In recent years, advertisers have established influencer tiers based on follower count: nano-nfluencers (1,000 to 9,999 followers), micro-influencers (10,000–99,999 followers), macro-influencers (100,000–999,999 followers), and mega-influencers (≥1.00 million followers; Wiley Citation2021).

While higher-tiered influencers (i.e., macro-influencers and mega-influencers) are regarded as more likable and popular than lower-tiered influencers (i.e., nano-influencers and micro-influencers), they are better suited for enhancing brand awareness across diverse audiences (Wiley Citation2021) and promoting products with standard designs (De Veirman, Cauberghe, and Hudders Citation2017). Meanwhile, lower-tiered influencers possess greater attitudinal and behavioral impact due to their intimate PSI with consumers (Wiley Citation2021). In fact, followers evaluated the product more positively and displayed greater purchase intentions when exposed to lower-tiered (vs. higher-tiered) influencers (Kay, Mulcahy, and Parkinson Citation2020; Park et al. Citation2021). Moreover, consumers perceived greater homophily with low-tiered (vs. high-tiered) influencers, regarding PSAs more favorably and displaying greater behavioral compliance (Looi et al. Citation2022).

Although these observations were contextualized among HIs, consumers may respond similarly to VIs. Since higher-tiered VIs emulate the aspirational lifestyles of higher-tiered HIs and their luxury brand endorsements (Ong Citation2020), consumers may perceive higher-tiered VIs as conduits for brands to maximize profits. Contrastingly, consumers share deeper relational bonds with lower-tiered HIs due to their homophily and relatability (Looi et al. Citation2022). Congruent with the CASA framework, consumers may extrapolate their perceptions of lower-tiered HIs to lower-tiered VIs. Thus, this study hypothesizes:

H1: For both HIs and VIs, Instagram posts from lower-tiered influencers will elicit significantly greater engagement than those from higher-tiered influencers.

Yet, limited research assessed whether VIs would widen or narrow gaps in engagement between higher-tiered and lower-tiered influencers. Consumers followed and interacted with VIs due to the visual appeal and mysteriousness associated with their technological novelty (Choudhry et al. Citation2022; Lou et al. Citation2023). Additionally, consumers were intrigued by VIs’ content creativity, entertainment value, and responsiveness (Choudhry et al. Citation2022; Lou et al. Citation2023). As such, consumers may express greater intention to engage with prominent VIs, widening engagement gaps between established and emerging VIs. Meanwhile, consumers may experience fatigue from the repetitive content and oversaturation of HIs (Bakhtiari Citation2020), decreasing their intention to engage with established and emerging HIs and resulting in reduced engagement gaps. Considering the scarcity of research, this study examines:

RQ4: How will influencer type interact with influencer tier to affect engagement?

Influencers are typically conferred verified badges as algorithmic validation for their online prominence and opinion leadership (Kowtun Citation2020). Moreover, social media verification implies that the influencer’s social media profile is authentic and legitimate (Kowtun Citation2020). Although limited research has evaluated the role of Instagram verification, consumers may perceive influencers who have it as more authentic and legitimate. Since the effectiveness of influencer endorsements rests upon authenticity (Lee and Eastin Citation2021), consumers will be more likely to engage with influencers with (vs. without) Instagram verification. Thus, this study posits:

H2: For both HIs and VIs, Instagram posts from influencers with Instagram verification will elicit significantly greater engagement than those without Instagram verification.

Since VIs are digital personalities that are artificially constructed (Stein et al. Citation2022), consumers may perceive VIs with Instagram verification as more legitimate. Conversely, consumers may construe VIs without Instagram verification as social bots, which are fully or semi-automated computer agents that mimic humans’ information behaviors, such as liking and commenting on social media posts (Assenmacher et al. Citation2020). Relative to HIs, VIs with (vs. without) Instagram verification may experience wider engagement gaps. Given the limited research in this domain, this study investigates:

RQ5: How will influencer type interact with Instagram verification to affect engagement?

Additionally, this study assesses how influencer tier and Instagram verification determine their persuasiveness. While extant literature has examined the role of influencer tier, few studies have determined its joint impact with Instagram verification. Therefore, this study asks:

RQ6: For both HIs and VIs, how will influencer tier interact with Instagram verification to affect engagement?

Method

Using the sample of 424 Instagram influencers from study 1, we conducted a natural experiment using a 2 (influencer type: HIs vs. VIs) × 4 (influencer tier: nano-influencer vs. micro-influencer vs. macro-influencer vs. mega-influencer) × 2 (Instagram verification: verified vs. unverified) between-subjects, fractional factorial design. Similar to study 1, influencer type was operationalized in terms of identity (group 1: HIs, n = 347; group 2: VIs, n = 77). We also adhered to advertisers’ categorization for influencer tier (e.g., Wiley Citation2021) by segregating influencers into four groups based on their follower count (group 1: mega-influencers, n = 86; group 2: macro-influencers, n = 108; group 3: micro-influencers, n = 129; group 4: nano-influencers, n = 103). Additionally, we operationalized Instagram verification based on whether the influencer possessed a verified badge (group 1: verified, n = 185; group 2: unverified, n = 239).

Analytical Approach

Prior to statistical analysis, we identified and excluded 31 positive outliers that yielded far greater engagement than the mean values (i.e., observations with a standardized residual >3), resulting in a final sample of 393 influencers. This approach mitigated type I errors, reducing the findings’ susceptibility to false positives. Congruent with Arsenyan and Mirowska’s (Citation2021) approach, coupled with study 2’s fractional factorial design, we conducted a series of two-way analyses of variance (ANOVAs) to analyze how influencer type, influencer tier, and Instagram verification affected engagement. We also corroborated the analyses using non-parametric tests (Kruskal–Wallis H test and Games–Howell test) when the assumption of normality was violated (i.e., Shapiro–Wilk test of normality, p < .05). The aforementioned statistical analyses were conducted using Kassambara’s (Citation2021) rstatix package in RStudio.

Results

Main Effect of Influencer Type

In addressing RQ3, a two-way ANOVA revealed that influencer type significantly affected engagement, F(1, 385) = 7.41, p < .01, ηp2 = 0.02. This observation was replicated with a Kruskal–Wallis H test, χ2(1) = 6.80, p < .01. HIs (M = 5.48, SD = 4.62) elicited significantly greater engagement than VIs (M = 3.90, SD = 3.61).

Main Effect of Influencer Tier

A two-way ANOVA indicated that influencer tier significantly affected engagement, F(3, 385) = 6.86, p < .001, ηp2 = 0.05. This finding was corroborated with a Kruskal–Wallis H test, χ2(3) = 18, p < .001. A Games–Howell post hoc test found that nano-influencers (M = 5.43, SD = 4.41) and mega-influencers (M = 6.73, SD = 4.78) elicited significantly greater engagement than did macro-influencers (M = 3.81, SD = 3.09). However, nano-influencers (M = 5.43, SD = 4.41) did not differ significantly from micro-influencers (M = 5.12, SD = 4.99) and mega-influencers (M = 6.73, SD = 4.78). Micro-influencers (M = 5.12, SD = 4.99) also did not differ significantly from macro-influencers (M = 3.81, SD = 3.09) and mega-influencers (M = 6.73, SD = 4.78). Thus, H1 was partially supported.

Main Effect of Instagram Verification

A two-way ANOVA revealed that Instagram verification did not significantly affect engagement, F(1, 386) = 1.43, p > .05, ηp2 = 0.00. This observation was corroborated using a Kruskal–Wallis H test, χ2(1) = .28, p > .05. Contrary to H2, influencers with Instagram verification (M = 4.90, SD = 4.15) failed to elicit significantly greater engagement than those without Instagram verification (M = 5.43, SD = 4.74).

Interaction Effect of Influencer Type and Influencer Tier

In addressing RQ4, a two-way ANOVA indicated that the interaction between influencer type and influencer tier failed to attain statistical significance, F(3, 385) = .80, p > .05, ηp2 = 0.01.

Interaction Effect of Influencer Type and Instagram Verification

For RQ5, a two-way ANOVA indicated that the interaction between influencer type and Instagram verification was non-significant, F(1, 389) = 1.14, p > .05, ηp2 = 0.00.

Interaction Effect of Instagram Verification and Influencer Tier

In addressing RQ6, a two-way ANOVA found that influencer tier significantly interacted with Instagram verification to affect engagement, F(2, 386) = 3.13, p < .05, ηp2 = 0.02. Among influencers with Instagram verification, Tukey honestly significant difference tests revealed that mega-influencers (M = 6.83, SD = 4.80) elicited significantly greater engagement than micro-influencers (M = 1.86, SD = 1.51) and macro-influencers (M = 3.54, SD = 2.64). Notably, micro-influencers (M = 5.61, SD = 5.14) without Instagram verification elicited significantly greater engagement than micro-influencers (M = 1.86, SD = 1.51) and macro-influencers (M = 3.54, SD = 2.64) with Instagram verification. Results of study 2 are displayed in .

Table 3. Two-way analysis of variance of influencer type and influencer tier on engagement.

Table 4. Two-way analysis of variance of influencer type and Instagram verification on engagement.

Table 5. Two-way analysis of variance of influencer tier and Instagram verification on engagement.

Overall, study 2 observed that HIs obtained significantly greater engagement than VIs. Notably, the findings also revealed a curvilinear relationship between influencer tier and engagement, whereby nano-influencers and mega-influencers elicited significantly greater engagement than macro-influencers did (). Contrary to predictions, the main effect of Instagram verification was non-significant for both HIs and VIs. Meanwhile, the interaction between influencer type and tier was non-significant. Influencer type also did not significantly interact with Instagram verification. However, there was a significant interaction effect for Instagram verification and influencer tier.

Figure 3. Interaction of influencer tier and Instagram verification on engagement.

Figure 3. Interaction of influencer tier and Instagram verification on engagement.

Discussion

This study compared the persuasion strategies and outcomes between HIs and VIs. Study 1 revealed that HIs’ and VIs’ Instagram posts primarily conveyed positive emotions. The corpus of Instagram posts for both HIs and VIs contained a greater proportion of words with positive (vs. negative) sentiment. However, VIs were more likely to incorporate words with positive sentiment in their Instagram posts than were HIs.

Overall, words with positive sentiment were more intense than words with negative sentiment. However, VIs’ Instagram posts possessed greater emotional intensity than those of HIs. Since VIs content is generated using AI (Thomas and Fowler Citation2021), it reflects and reinforces the asymmetry of positive content on social media. The positivity bias also supported prior research, whereby individuals posted disproportionately more content conveying positive emotions and personal achievements (Reinecke and Trepte Citation2014; Utz Citation2015). It could also be attributed to prevailing norms of politeness on social media, where posting offensive content is construed as being rude and disrespectful (Spottswood and Hancock Citation2016). Hence, HIs and VIs may post positive content as impression management and/or in adherence to the norms of civic-mindedness on social media.

Study 1 also compared key themes that emerged from Instagram posts of HIs and VIs. While both HIs and VIs asserted expertise within niche domains, they established opinion leadership differently. VIs explicitly established their identity as influencers or models, while HIs implicitly established opinion leadership by demonstrating their ability to create content and influence informational flows. These different persuasion strategies could be attributed to HIs’ ubiquity, in contrast with VIs’ novelty.

Study 2 compared whether VIs elicited greater engagement on Instagram than did HIs. Congruent with the uncanny valley hypothesis (Mori, MacDorman, and Kageki Citation2012), HIs elicited significantly greater engagement than VIs did (Arsenyan and Mirowska Citation2021). These findings supported prior research (Green et al. Citation2008; MacDorman et al. Citation2009), whereby participants expressed greater agreement after encountering an attractive human face than an artificial face created using computer graphics. These uncanny valley effects could be attributed to perceptually incompatible visual stimuli (Kätsyri et al. Citation2015), including VIs’ interaction with HIs, or from embedding VIs in real environments (i.e., mixed reality; Ham et al. Citation2023).

Overall, the uncanny valley hypothesis emerged as the dominant framework for predicting VIs’ persuasiveness. If the CASA framework prevailed, VIs’ engagement would not significantly differ from that of HIs. If individuals relied on machine heuristics, VIs would obtain significantly greater engagement than HIs did. Therefore, future research should replicate these findings by juxtaposing the aforementioned theories to determine the mechanisms underlying VIs’ persuasiveness.

Additionally, study 2 assessed how influencer tier determined engagement on Instagram. Similar to the findings of Kay, Mulcahy, and Parkinson (Citation2020), nano-influencers received significantly greater engagement than did macro-influencers. Contrastingly, mega-influencers also elicited significantly greater engagement than did macro-influencers, supporting research by De Veirman et al. (Citation2017). Since influencer tier did not directly correspond with engagement, future studies may verify this curvilinear relationship across other persuasion outcomes, including consumers’ brand awareness, brand attitudes, and purchase intention.

Instagram verification did not significantly affect engagement independently or in combination with influencer type. However, there was a significant and disordinal interaction between influencer tier and Instagram verification. Among influencers with verified Instagram profiles, mega-influencers received significantly more engagement than micro-influencers and macro-influencers did. Yet, micro-influencers without Instagram verification outperformed micro-influencers and macro-influencers with Instagram verification. Altogether, follower count did not directly correspond with engagement for influencers without Instagram verification. However, follower count corresponded directly with engagement among influencers with Instagram verification. Thus, findings suggested distinct persuasive mechanisms between lower-tiered and higher-tiered influencers, which supported recent research (Kay, Mulcahy, and Parkinson Citation2020; Looi et al. Citation2022; Park et al. Citation2021).

Study Implications

This study provided several research and practical contributions. By extending human–AI interaction to influencer marketing, this study clarified scholarly understanding about VIs. Additionally, this study attested to the uncanny valley in determining VIs’ persuasiveness. In doing so, this study also calls for further research to assess the theoretical boundaries of CASA and machine heuristics. This study also advanced influencer marketing research by demonstrating that HIs elicited significantly greater engagement on Instagram. As such, VIs should not be construed as perfect substitutes for HIs.

Beyond considering influencer type, this study utilized a nuanced delineation and enhanced existing operationalizations for influencer tier. Moreover, this study evaluated the under-studied component of Instagram verification, which significantly interacted with influencer tier to shape engagement. Notably, only higher-tiered influencers asserted opinion leadership using follower count and Instagram verification. Therefore, the findings supported recent research regarding the differing persuasive mechanisms of higher-tiered and lower-tiered influencers (Kay, Mulcahy, and Parkinson Citation2020; Looi et al. Citation2022; Park et al. Citation2021).

Practically, our findings provide insights for advertisers regarding the selection of influencers: Instead of succumbing to VIs’ hype and technological novelty, advertisers should feature HIs to elicit greater engagement on Instagram. Advertisers should also pay attention to influencer tier and Instagram verification when selecting a campaign spokesperson. If advertisers intend to enhance consumers’ awareness and recognition of brands and products, they should feature mega-influencers with Instagram verification. However, if advertisers wish to influence consumers’ attitudes and behaviors, they should feature nano-influencers without Instagram verification. Since uncanny valley effects may arise when consumers encounter humanlike VIs, the findings also suggest that AI experts should create VIs modeled after inanimate objects (e.g., @thegoodadvicecupcake) or anime-like appearances (e.g., @noonoouri).

Limitations and Future Research

This study possesses several limitations that may contextualize interpretations of its findings. First, this study focused on humanlike VIs, which may constrain the findings’ external validity to VIs with nonhuman or anime-like avatars. Therefore, future research may replicate this study by juxtaposing VIs with varying humanlikeness. Second, this study exclusively examined Instagram data since it is most frequently used by VIs. However, future research is needed to determine the findings’ generalizability across different social media (e.g., YouTube, TikTok). Third, study 1 conducted dictionary-based sentiment analysis, which is susceptible to methodological issues, including “insufficient coverage of sentiment words and lack of domain words” (Xu et al. Citation2019). Dictionary-based sentiment analysis may also inaccurately detect the sentiment of words conveying sarcasm (Dhaoui, Webster, and Tan Citation2017; Guo et al. Citation2016). Moreover, dictionary-based sentiment analysis does not account for common misspellings and internet lingo. Although this study sought to enhance the accuracy of sentiment scoring by corroborating the findings across multiple lexicons (AFINN and Bing), computational researchers noted the superior accuracy of sentiment analysis conducted with ML (Guo et al. Citation2016) and deep-learning approaches (Al-Qablan et al. Citation2023; Chauhan et al. Citation2023; Xu et al. Citation2019). Therefore, future research should verify the generalizability of these observations by comparing the accuracy of sentiment analysis approaches (dictionary-based, ML, deep-learning) for categorizing emojis, images, and text from Instagram posts. Fourth, this study utilized a natural experiment to compare the persuasiveness of HIs and VIs. Despite its ecological validity, this method possesses less internal validity than online and laboratory experiments. Additionally, data regarding engagement followed a non-normal distribution. Considering these violations to the normality assumption, study 2 could not conduct linear regressions to empirically assess influencer tier and persuasion strategies (emotional valence, emotional intensity, themes) as predictors of engagement. Therefore, future research may corroborate this study’s observations with online experiments. Congruent with Lou and Yuan’s (Citation2019) analytical approach, future studies may utilize structural equation modeling to juxtapose how influencers’ source characteristics and content attributes jointly determine persuasion. Furthermore, data regarding engagement may not adequately reflect consumers’ PSI with HIs and VIs. For instance, consumers may “hate-watch” to derive enjoyment from criticizing or mocking content on social media (Travers Citation2023). Consumers may also comment on controversial Instagram posts to voice their disagreement. Hence, future research should triangulate engagement data with content analysis of user comments to determine the emotional valence, emotional intensity, and type of consumers’ imagined relational bonds with HIs and VIs. Fifth, this study did not control for participants’ perceived familiarity with VIs. Hence, future studies should analyze consumers’ perceived familiarity with VIs as control or moderating variables in shaping attitudes and behaviors. Finally, this study included an unequal sample of HIs and VIs. While this approach enhances the comparability of HIs and VIs, it may decrease the study’s statistical power and increase its type I error rate (Rusticus and Lovato Citation2019). Therefore, future research may verify the findings using experiments featuring fictitious HIs and VIs to ensure that they possess comparable follower count, Instagram verification, and demographics (age, ethnicity, gender). More importantly, researchers will have greater control over the sample size and equivalence of participants across conditions.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes on contributors

Jiemin Looi

Jiemin Looi (Ph.D., The University of Texas at Austin) is an assistant professor at the School of Communication, Hong Kong Baptist University.

Lee Ann Kahlor

Lee Ann Kahlor (Ph.D., University of Wisconsin-Madison) is a professor in the Stan Richards School of Advertising and Public Relations, Moody College of Communication, The University of Texas at Austin.

References

  • Al-Qablan, T. A., M. H. Mohd Noor, M. A. Al-Betar, and A. T. Khader. 2023. “A Survey on Sentiment Analysis and Its Applications.” Neural Computing and Applications 35 (29): 21567–21601. https://doi.org/10.1007/s00521-023-08941-y.
  • Arsenyan, J., and A. Mirowska. 2021. “Almost Human? A Comparative Case Study on the Social Media Presence of Virtual Influencers.” International Journal of Human-Computer Studies 155: 102694. https://doi.org/10.1016/j.ijhcs.2021.102694.
  • Assenmacher, D., L. Clever, L. Frischlich, T. Quandt, H. Trautmann, and C. Grimme. 2020. “Demystifying Social Bots: On the Intelligence of Automated Social Media Actors.” Social Media + Society 6 (3): 205630512093926. https://doi.org/10.1177/2056305120939264.
  • Bakhtiari, K. 2020, April 6. “Influencer Fatigue Sets Stage for a New Generation of Creators.” Forbes. https://www.forbes.com/sites/kianbakhtiari/2020/04/06/influencer-fatigue-sets-the-stage-for-a-new-generation-of-creators/?sh=14c1fed12535
  • Bradley, S. 2021, March 20. “Even Better Than the Real Thing? Meet the Virtual Influencers Taking over Your Feeds.” The Drum. https://www.thedrum.com/news/2020/03/20/even-better-the-real-thing-meet-the-virtual-influencers-taking-over-your-feeds
  • Brandwatch. 2020. “Crimson Hexagon: ForSight User Guide.” https://www.brandwatch.com/wp-content/uploads/2020/10/Crimson-Hexagon-ForSight-User-Guide.pdf
  • Brooks, G., J. Drenten, and M. J. Piskorski. 2021. “Influencer Celebrification: How Social Media Influencers Acquire Celebrity Capital.” Journal of Advertising 50 (5): 528–547. https://doi.org/10.1080/00913367.2021.1977737.
  • Burnell, K., M. J. George, A. R. Kurup, M. K. Underwood, and R. A. Ackerman. 2021. “Associations between Self-Reports and Device-Reports of Social Networking Site Use: An Application of the Truth and Bias Model.” Communication Methods and Measures 15 (2): 156–163. https://doi.org/10.1080/19312458.2021.1918654.
  • Chacon, B. 2018, December 9. “How to Calculate Your Instagram Engagement Rate.” Later. https://later.com/blog/instagram-engagement-rate/
  • Chauhan, G. S., R. Nahta, Y. K. Meena, and D. Gopalani. 2023. “Aspect Based Sentiment Analysis Using Deep Learning Approaches: A Survey.” Computer Science Review 49: 100576. https://doi.org/10.1016/j.cosrev.2023.100576.
  • Choudhry, A., J. Han, X. Xu, and Y. Huang. 2022. ““I Felt a Little Crazy following a ‘Doll’” Investigating Real Influence of Virtual Influencers on Their Followers.” Proceedings of the ACM on Human-Computer Interaction 6 (GROUP): 1–28. https://doi.org/10.1145/3492862.
  • Corporate Finance Institute. 2022, April 29. “Engagement Rate: The Level of Engagement Generated from a Created Content or a Brand Campaign.” https://corporatefinanceinstitute.com/resources/knowledge/ecommerce-saas/engagement-rate/
  • De Veirman, M., V. Cauberghe, and L. Hudders. 2017. “Marketing through Instagram Influencers: The Impact of Number of Followers and Product Divergence on Brand Attitude.” International Journal of Advertising 36 (5): 798–828. https://doi.org/10.1080/02650487.2017.1348035.
  • Demirel, A. 2020. “An Examination of a Campaign Hashtag (# OptOutside) with Google Trends and Twitter.” Journal of Interactive Advertising 20 (3): 165–180. https://doi.org/10.1080/15252019.2020.1840460.
  • Demirel, A. 2022. “Voluntary Simplicity: An Exploration through Text Analysis.” International Journal of Consumer Studies 46 (1): 75–87. https://doi.org/10.1111/ijcs.12644.
  • Dhaoui, C., C. M. Webster, and L. P. Tan. 2017. “Social Media Sentiment Analysis: Lexicon versus Machine Learning.” Journal of Consumer Marketing 34 (6): 480–488. https://doi.org/10.1108/JCM-03-2017-2141.
  • Dibble, J. L., T. Hartmann, and S. F. Rosaen. 2016. “Parasocial Interaction and Parasocial Relationship: Conceptual Clarification and a Critical Assessment of Measures.” Human Communication Research 42 (1): 21–44. https://doi.org/10.1111/hcre.12063.
  • Downer, K., C. Wells, and C. Crichton. 2019. “All Work and No Play: A Text Analysis.” International Journal of Market Research 61 (3): 236–251. https://doi.org/10.1177/1470785318821849.
  • Franke, C., A. Groeppel-Klein, and K. Müller. 2022. “Consumers’ Responses to Virtual Influencers as Advertising Endorsers: Novel and Effective or Uncanny and Deceiving?” Journal of Advertising 52 (4): 523–539. https://doi.org/10.1080/00913367.2022.2154721.
  • Gambino, A., J. Fox, and R. A. Ratan. 2020. “Building a Stronger CASA: Extending the Computers Are Social Actors Paradigm.” Human-Machine Communication 1: 71–86. https://doi.org/10.30658/hmc.1.5.
  • Giles, D. C. 2002. “Parasocial Interaction: A Review of the Literature and a Model for Future Research.” Media Psychology 4 (3): 279–305. https://doi.org/10.1207/S1532785XMEP0403_04.
  • Gong, W., and X. Li. 2017. “Engaging Fans on Microblog: The Synthetic Influence of Parasocial Interaction and Source Characteristics on Celebrity Endorsement.” Psychology & Marketing 34 (7): 720–732. https://doi.org/10.1002/mar.21018.
  • Green, R. D., K. F. MacDorman, C. C. Ho, and S. Vasudevan. 2008. “Sensitivity to the Proportions of Faces That Vary in Human Likeness.” Computers in Human Behavior 24 (5): 2456–2474. https://doi.org/10.1016/j.chb.2008.02.019.
  • Guo, L., C. J. Vargo, Z. Pan, W. Ding, and P. Ishwar. 2016. “Big Social Data Analytics in Journalism and Mass Communication: Comparing Dictionary-Based Text Analysis and Unsupervised Topic Modeling.” Journalism & Mass Communication Quarterly 93 (2): 332–359. https://doi.org/10.1177/1077699016639231.
  • Ham, J., S. Li, P. Shah, and M. S. Eastin. 2023. “The “Mixed” Reality of Virtual Brand Endorsers: Understanding the Effect of Brand Engagement and Social Cues on Technological Perceptions and Advertising Effectiveness.” Journal of Interactive Advertising 23 (2): 98–113. https://doi.org/10.1080/15252019.2023.2185557.
  • Hartmann, T., and C. Goldhoorn. 2011. “Horton and Wohl Revisited: Exploring Viewers’ Experience of Parasocial Interaction.” Journal of Communication 61 (6): 1104–1121. https://doi.org/10.1111/j.1460-2466.2011.01595.x.
  • Hayes, J. L., B. C. Britt, W. Evans, S. W. Rush, N. A. Towery, and A. C. Adamson. 2020. “Can Social Media Listening Platforms’ Artificial Intelligence Be Trusted? Examining the Accuracy of Crimson Hexagon’s (Now Brandwatch Consumer Research’s) AI-Driven Analyses.” Journal of Advertising 50 (1): 81–91. https://doi.org/10.1080/00913367.2020.1809576.
  • Hiort, A. 2022, February 15. “Yes, Virtual Influencers Are Taking Jobs. Virtual Humans. https://www.virtualhumans.org/article/yes-virtual-influencers-are-taking-jobs
  • Horton, D., and R. R. Wohl. 1956. “Mass Communication and Parasocial Interaction: Observations on Intimacy at a Distance.” Psychiatry 19 (3): 215–229. https://doi.org/10.1080/00332747.1956.11023049.
  • Kassambara, A. 2021. “rstatix: Pipe-Friendly Framework for Basic Statistical Tests.” R package version 0.7.0. https://CRAN.R-project.org/package=rstatix
  • Kätsyri, J., K. Förger, M. Mäkäräinen, and T. Takala. 2015. “A Review of Empirical Evidence on Different Uncanny Valley Hypotheses: Support for Perceptual Mismatch as One Road to the Valley of Eeriness.” Frontiers in Psychology 6: 390. https://doi.org/10.3389/fpsyg.2015.00390.
  • Kay, S., R. Mulcahy, and J. Parkinson. 2020. “When Less Is More: The Impact of Macro and Micro Social Media Influencers’ Disclosure.” Journal of Marketing Management 36 (3-4): 248–278. https://doi.org/10.1080/0267257X.2020.1718740.
  • Kietzmann, J., J. Paschen, and E. Treen. 2018. “Artificial Intelligence in Advertising: How Marketers Can Leverage Artificial Intelligence along the Consumer Journey.” Journal of Advertising Research 58 (3): 263–267. https://doi.org/10.2501/JAR-2018-035.
  • Kim, H. 2022. “Keeping up with Influencers: Exploring the Impact of Social Presence and Parasocial Interactions on Instagram.” International Journal of Advertising 41 (3): 414–434. https://doi.org/10.1080/02650487.2021.1886477.
  • Kim, J., and H. Song. 2016. “Celebrity’s Self-Disclosure on Twitter and Parasocial Relationships: A Mediating Role of Social Presence.” Computers in Human Behavior 62: 570–577. https://doi.org/10.1016/j.chb.2016.03.083.
  • Kowtun, A. 2020, December 9. “How to Actually Get Verified on Social Media.” Forbes. https://www.forbes.com/sites/forbesbusinessdevelopmentcouncil/2020/12/09/how-to-actually-get-verified-on-social-media/?sh=47b8b0004553
  • Lee, D., and C. D. Ham. 2023. “AI versus Human: Rethinking the Role of Agent Knowledge in Consumers’ Coping Mechanism Related to Influencer Marketing.” Journal of Interactive Advertising 23 (3): 241–258. https://doi.org/10.1080/15252019.2023.2217830.
  • Lee, J. A., and M. S. Eastin. 2021. “Perceived Authenticity of Social Media Influencers: Scale Development and Validation.” Journal of Research in Interactive Marketing 15 (4): 822–841. https://doi.org/10.1108/JRIM-12-2020-0253.
  • Lee, J. A., S. Sudarshan, K. L. Sussman, L. F. Bright, and M. S. Eastin. 2022. “Why Are Consumers following Social Media Influencers on Instagram? Exploration of Consumers’ Motives for Following Influencers and the Role of Materialism.” International Journal of Advertising 41 (1): 78–100. https://doi.org/10.1080/02650487.2021.1964226.
  • Lewis, D. D., Y. Yang, T. Russell-Rose, and F. Li. 2004. “RCV1: A New Benchmark Collection for Text Categorization Research.” Journal of Machine Learning Research 5 (Apr): 361–397.
  • Liu, X., A. C. Burns, and Y. Hou. 2017. “An Investigation of Brand-Related User-Generated Content on Twitter.” Journal of Advertising 46 (2): 236–247. https://doi.org/10.1080/00913367.2017.1297273.
  • Looi, J., D. Kemp, and Y. W. G. Song. 2022. “Instagram Influencers in Health Communication: Examining the Roles of Influencer Tier and Message Construal in COVID-19-Prevention Public Service Announcements.” Journal of Interactive Advertising 23 (1): 14–32. https://doi.org/10.1080/15252019.2022.2140316.
  • Lou, C. 2022. “Social Media Influencers and Followers: Theorization of a Trans-Parasocial Relation and Explication of Its Implications for Influencer Advertising.” Journal of Advertising 51 (1): 4–21. https://doi.org/10.1080/00913367.2021.1880345.
  • Lou, C., and S. Yuan. 2019. “Influencer Marketing: How Message Value and Credibility Affect Consumer Trust of Branded Content on Social Media.” Journal of Interactive Advertising 19 (1): 58–73. https://doi.org/10.1080/15252019.2018.153350.
  • Lou, C., S. S. Tan, and X. Chen. 2019. “Investigating Consumer Engagement with Influencer-vs. Brand-Promoted Ads: The Roles of Source and Disclosure.” Journal of Interactive Advertising 19 (3): 169–186. https://doi.org/10.1080/15252019.2019.1667928.
  • Lou, C., S. T. J. Kiew, T. Chen, T. Y. M. Lee, J. E. C. Ong, and Z. Phua. 2023. “Authentically Fake? How Consumers Respond to the Influence of Virtual Influencers.” Journal of Advertising 52 (4): 540–557. https://doi.org/10.1080/00913367.2022.2149641.
  • MacDorman, K. F., R. D. Green, C. C. Ho, and C. T. Koch. 2009. “Too Real for Comfort? Uncanny Responses to Computer Generated Faces.” Computers in Human Behavior 25 (3): 695–710. https://doi.org/10.1016/j.chb.2008.12.026.
  • Maier, D., A. Waldherr, P. Miltner, G. Wiedemann, A. Niekler, A. Keinert, B. Pfetsch, G. Heyer, U. Reber, T. Häussler, et al. 2018. “Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology.” Communication Methods and Measures 12 (2-3): 93–118. https://doi.org/10.1080/19312458.2018.1430754.
  • Mori, M., K. F. MacDorman, and N. Kageki. 2012. “The Uncanny Valley [From the Field].” IEEE Robotics & Automation Magazine 19 (2): 98–100. https://doi.org/10.1109/MRA.2012.2192811.
  • Moustakas, E., N. Lamba, D. Mahmoud, and C. Ranganathan. 2020, June. “Blurring Lines between Fiction and Reality: Perspectives of Experts on Marketing Effectiveness of Virtual Influencers.” In 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security), Dublin, Ireland, 1–6. IEEE. https://doi.org/10.1109/CyberSecurity49315.2020.9138861.
  • O’Halloran, S., M. Dumas, S. Maskey, G. McAllister, and D. K. Park. 2017. “Computational Data Sciences and the Regulation of Banking and Financial Services.” In From Social Data Mining and Analysis to Prediction and Community Detection, edited by Mehmet Kaya, Özcan Erdoǧan, and Jon Rokne, 179–209. Cham: Springer.
  • Ong, T. 2020, October 29. “Virtual Influencers Make Real Money While COVID Locks Down Human Stars.” Bloomberg Businessweek. https://www.bloomberg.com/news/features/2020-10-29/lil-miquela-lol-s-seraphine-virtual-influencers-make-more-real-money-than-ever
  • Park, J., J. M. Lee, V. Y. Xiong, F. Septianto, and Y. Seo. 2021. “David and Goliath: When and Why Micro-Influencers Are More Persuasive than Mega-Influencers.” Journal of Advertising 50 (5): 584–602. https://doi.org/10.1080/00913367.2021.1980470.
  • Puschmann, C., and A. Powell. 2018. “Turning Words into Consumer Preferences: How Sentiment Analysis Is Framed in Research and the News Media.” Social Media + Society 4 (3): 205630511879772. https://doi.org/10.1177/2056305118797724.
  • Reinecke, L., and S. Trepte. 2014. “Authenticity and Well-Being on Social Network Sites: A Two-Wave Longitudinal Study on the Effects of Online Authenticity and the Positivity Bias in SNS Communication.” Computers in Human Behavior 30: 95–102. https://doi.org/10.1016/j.chb.2013.07.030.
  • Roberts, M. E., B. M. Stewart, and D. Tingley. 2019. “Stm: An R Package for Structural Topic Models.” Journal of Statistical Software 91 (2): 1–40. https://doi.org/10.18637/jss.v091.i02.
  • Rosenthal-Von Der Pütten, A. M., and N. C. Krämer. 2014. “How Design Characteristics of Robots Determine Evaluation and Uncanny Valley Related Responses.” Computers in Human Behavior 36: 422–439. https://doi.org/10.1016/j.chb.2014.03.066.
  • Rubin, A. M., E. M. Perse, and R. A. Powell. 1985. “Loneliness, Parasocial Interaction, and Local Television News Viewing.” Human Communication Research 12 (2): 155–180. https://doi.org/10.1111/j.1468-2958.1985.tb00071.x.
  • Rusticus, S. A., and C. Y. Lovato. 2019. “Impact of Sample Size and Variability on the Power and Type I Error Rates of Equivalence Tests: A Simulation Study.” Practical Assessment, Research, and Evaluation 19 (1): 11.
  • Schramm, H., and T. Hartmann. 2008. “The PSI-Process Scales. A New Measure to Assess the Intensity and Breadth of Parasocial Processes.” COMM 33 (4): 385–401. https://doi.org/10.1515/COMM.2008.025.
  • Silge, J., and D. Robinson. 2016. “Tidytext: Text Mining and Analysis Using Tidy Data Principles in R.” The Journal of Open Source Software 1 (3): 37. https://doi.org/10.21105/joss.00037.
  • Snowball. n.d. “An English Stop Word List.” http://snowball.tartarus.org/algorithms/english/stop.txt
  • Spottswood, E. L., and J. T. Hancock. 2016. “The Positivity Bias and Prosocial Deception on Facebook.” Computers in Human Behavior 65: 252–259. https://doi.org/10.1016/j.chb.2016.08.019.
  • Stein, J. P., P. Linda Breves, and N. Anders. 2022. “Parasocial Interactions with Real and Virtual Influencers: The Role of Perceived Similarity and Human-Likeness.” New Media & Society : 146144482211029. https://doi.org/10.1177/14614448221102900.
  • Sundar, S. S. 2008. “The MAIN Model: A Heuristic Approach to Understanding Technology Effects on Credibility.” In Digital Media, Youth, and Credibility, edited by Miriam J. Metzger and Andrew J. Flanagin, 73–100. Cambridge, MA: The MIT Press. https://doi.org/10.1162/dmal.9780262562324.073.
  • Thomas, V. L., and K. Fowler. 2021. “Close Encounters of the AI Kind: Use of AI Influencers as Brand Endorsers.” Journal of Advertising 50 (1): 11–25. https://doi.org/10.1080/00913367.2020.1810595.
  • Tian, L., C. Lai, and J. D. Moore. 2018. “Polarity and Intensity: The Two Aspects of Sentiment Analysis.” arXiv preprint arXiv: 1807.01466. https://doi.org/10.48550/arXiv.1807.01466.
  • Travers, M. 2023, September 22. “A Psychologist Explains Why We ‘Hate-Watch’ Cringe TV.” Forbes. https://www.forbes.com/sites/traversmark/2023/09/22/a-psychologist-explains-why-we-hate-watch-cringe-tv/?sh=72e052c795ba
  • Utz, S. 2015. “The Function of Self-Disclosure on Social Network Sites: Not Only Intimate, but Also Positive and Entertaining Self-Disclosures Increase the Feeling of Connection.” Computers in Human Behavior 45: 1–10. https://doi.org/10.1016/j.chb.2014.11.076.
  • Wang, A. 2006. “Advertising Engagement: A Driver of Message Involvement on Message Effects.” Journal of Advertising Research 46 (4): 355–368. https://doi.org/10.2501/S0021849906060429.
  • Wiley, D. 2021, February 4. “Influencer Marketing’s Surprising Rise of the “Everyperson”. Forbes. https://www.forbes.com/sites/forbesagencycouncil/2021/02/04/influencer-marketings-surprising-rise-of-the-everyperson/?sh=3373ef163b23
  • Xu, G., Y. Meng, X. Qiu, Z. Yu, and X. Wu. 2019. “Sentiment Analysis of Comment Texts Based on BiLSTM.” IEEE Access 7: 51522–51532. https://doi.org/10.1109/ACCESS.2019.2909919.
  • Yang, J., P. Chuenterawong, H. Lee, Y. Tian, and T. M. Chock. 2023. “Human versus Virtual Influencer: The Effect of Humanness and Interactivity on Persuasive CSR Messaging.” Journal of Interactive Advertising 23 (3): 275–292. https://doi.org/10.1080/15252019.2023.2189036.
  • Yencken, L. [@larsyencken]. 2011. “stopwords.txt.” Freely Available Stopword List, Balancing Coverage and Size [Data Frame]. GitHub. https://gist.github.com/larsyencken/1440509?permalink_comment_id=3559938
  • Yun, J. T., C. M. Segijn, S. Pearson, E. C. Malthouse, J. A. Konstan, and V. Shankar. 2020. “Challenges and Future Directions of Computational Advertising Measurement Systems.” Journal of Advertising 49 (4): 446–458. https://doi.org/10.1080/00913367.2020.1795757.
  • Zhang, L., S. Wang, and B. Liu. 2018. “Deep Learning for Sentiment Analysis: A Survey.” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8 (4): 1253. https://doi.org/10.1002/widm.1253.