Discrete Emotion Synchronicity and Video Engagement on Social Media: A Moment-to-Moment Analysis

ABSTRACT

Although previous studies have identified several discrete emotion features that drive video diffusion on social media, little research has viewed the video consumption experience as a continuous communication process and combined viewer-side moment-to-moment (MTM) emotional reactions with supply-side MTM emotional dynamics. In this article, we examine how discrete emotion synchronicity, which refers to the synchronicity between MTM discrete emotions in the video content and MTM viewer emotional reactions, is associated with user engagement on video-based social media. Using a set of state-of-the-art deep learning algorithms and a regression model, we first extract discrete emotion synchronicity from videos with multimodal data streams. Then, we conduct an empirical analysis to determine the impact of discrete emotion synchronicity on user engagement. Our results indicate that discrete emotion synchronicity does not always lead to higher user engagement on video-based social media. First, discrete emotion synchronicity that increases the number of video tips (i.e., joy) cannot increase the number of saves and shares. Second, discrete emotion synchronicity (i.e., calm, solemnity, and sadness) has a negative effect on 3 types of engagement behaviors in some cases. Third, the negative effect of discrete emotion synchronicity outweighs its positive effect. Our work offers new insight into how discrete emotions facilitate or hinder user engagement in the context of video-based social media and has practical implications for firms’ marketing strategies on video-oriented social media.

KEYWORDS AND PHRASES:

• Discrete emotions
• moment-to-moment synchronicity
• user engagement
• multimodal data
• video-based social media

Acknowledgments

We thank the Editor-in-Chief and the anonymous reviewers for their constructive comments and suggestions. This work is supported by the National Natural Science Foundation of China (grant 72201269) and the School of Interdisciplinary Studies, Renmin University of China.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. https://github.com/Zulko/moviepy

2. http://ffmpeg.org/

3. https://image.baidu.com

4. The robustness check of the data filtering strategy is provided in Appendix B. As Table B1 shows, when using all data instead of filtered data, the results remain similar.

Additional information

Funding

This work was supported by the School of Interdisciplinary Studies, Renmin University of China; National Natural Science Foundation of China [72201269].

Notes on contributors

Dinghao Xi

Dingho Xi ([email protected]) is a Ph.D. student at the School of Information, Renmin University of China. His research interests include social media and business intelligence. His research has been published in Information Processing & Management.

Jilei Zhou

Jilei Zhou ([email protected]; corresponding author) is an assistant professor at the School of Information, Renmin University of China. She obtained her Ph.D. from Peking University, China. Her research interests include management information systems and business intelligence. Dr. Zhou’s research has been published in Decision Support Systems and Electronic Commerce Research and Applications.

Wei Xu

Wei Xu ([email protected]) is a professor at the School of Information, Renmin University of China. He received her Ph.D. from the Chinese Academy of Sciences. His research interests include business analysis and social media. Dr. Xu’s research has been published in Information Systems Research and Production and Operations Management.

Liumin Tang

Liumin Tang ([email protected]) is a master’s student at the School of Information, Renmin University of China, majoring in management science and engineering. Her research interests include business analysis, social media, and deep learning.