Discrete Emotion Synchronicity and Video Engagement on Social Media: A Moment-to-Moment Analysis

References

• Adams, B.; Dorai, C.; and Venkatesh, S. Toward automatic extraction of expressive elements from motion pictures: Tempo. IEEE Transactions on Multimedia, 4, 4 (2002), 472–481.
   Web of Science ®Google Scholar
• Ahmad, S.N.; and Laroche, M. How do expressed emotions affect the helpfulness of a product review? Evidence from reviews using latent semantic analysis. International Journal of Electronic Commerce, 20, 1 (2015), 76–111.
   Web of Science ®Google Scholar
• Alex, S. TikTok reveals detailed user numbers for the first time. 2020. https://www.cnbc.com/2020/08/24/tiktok-reveals-us-global-user-growth-numbers-for-first-time.html (accessed on June 5, 2023).
   Google Scholar
• Alhabash, S.; and McAlister, A.R. Redefining virality in less broad strokes: Predicting viral behavioral intentions from motivations and uses of Facebook and Twitter. New Media & Society, 17, 8 (2015), 1317–1339.
   Web of Science ®Google Scholar
• Aljanaki, A.; Wiering, F.; and Veltkamp, R.C. Studying emotion induced by music through a crowdsourcing game. Information Processing & Management, 52, 1 (2016), 115–128.
   Web of Science ®Google Scholar
• Apostolidis, E.; and Mezaris, V. Fast shot segmentation combining global and local visual descriptors. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 6583–6587.
   Google Scholar
• Barnett, S.B.; and Cerf, M. A ticket for your thoughts: Method for predicting content recall and sales using neural similarity of moviegoers. Journal of Consumer Research, 44, 1 (2017), 160–181.
   Web of Science ®Google Scholar
• Barrett, L.F.; and Russell, J.A. The structure of current affect: Controversies and emerging consensus. Current Directions in Psychological Science, 8, 1 (1999), 10–14.
   Web of Science ®Google Scholar
• Baumeister, R.F.; Bratslavsky, E.; Finkenauer, C.; and Vohs, K.D. Bad is stronger than good. Review of General Psychology, 5, 4 (2001), 323–370.
   Google Scholar
• Belch, G.E.; Belch, M.A.; Guolla, M.A.; Webb-Hughes, A.M.; and Skolnick, H. Advertising and Promotion: An Integrated Marketing Communications Perspective, Vol. 6. New York: McGraw-Hill/Irwin, 2004.
   Google Scholar
• Berger, J. Arousal increases social transmission of information. Psychological Science, 22, 7 (2011), 891–893.
   PubMed Web of Science ®Google Scholar
• Berger, J.; and Milkman, K. L. What makes online content viral? Journal of Marketing Research, 49, 2 (2012), 192–205.
   Web of Science ®Google Scholar
• Berger, J.; Kim, Y.D.; and Meyer, R. What makes content engaging? How emotional dynamics shape success. Journal of Consumer Research, 48, 2 (2021), 235–250.
   Web of Science ®Google Scholar
• Berger, J.; Moe, W.W.; and Schweidel, D.A. What holds attention? Linguistic drivers of engagement. Journal of Marketing, 87, 5 (2023), 793–809.
   Web of Science ®Google Scholar
• Bordwell, D.; Thompson, K.; and Smith, J. Film Art: An Introduction (Vol. 7). New York: McGraw-Hill, 1993.
   Google Scholar
• Bradley, M.M.; and Lang, P.J. Affective reactions to acoustic stimuli. Psychophysiology, 37, 2 (2000), 204–215.
   PubMed Web of Science ®Google Scholar
• Brodie, R. J.; Ilic, A.; Juric, B.; and Hollebeek, L. Consumer engagement in a virtual brand community: An exploratory analysis. Journal of Business Research, 66, 1 (2013), 105–114.
   Web of Science ®Google Scholar
• Buechel, E.C.; and Berger, J. Microblogging and the value of undirected communication. Journal of Consumer Psychology, 28, 1 (2018), 40–55.
   Web of Science ®Google Scholar
• Buehlmann, A.; and Deco, G. Optimal information transfer in the cortex through synchronization. PLoS Computational Biology, 6, 9 (2010), e1000934.
   PubMed Web of Science ®Google Scholar
• Caschera, M.C.; Grifoni, P.; and Ferri, F. Emotion classification from speech and text in videos using a multimodal approach. Multimodal Technologies and Interaction, 6, 4 (2022), 28.
   Google Scholar
• Chandrasekaran, G.; Nguyen, T.N.; and Hemanth, D.J. Multimodal sentimental analysis for social media applications: A comprehensive review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11 (2021), 1–28.
   Web of Science ®Google Scholar
• Chen, Y.; Wang, Y.; Zhang, J.; and Wang, J. An estimation of online video user engagement from features of continuous emotions. IEEE Transactions on Affective Computing, 2021.
   Google Scholar
• Chikkerur, S.; Sundaram, V.; Reisslein, M.; and Karam, L.J. Objective video quality assessment methods: A classification, review, and performance comparison. IEEE Transactions on Broadcasting, 57, 2 (2011), 165–182.
   Web of Science ®Google Scholar
• Cowen, A.S.; and Keltner, D. Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences, 114, 38 (2017), E7900–E7909.
   Google Scholar
• Cutting, J.E. Narrative theory and the dynamics of popular movies. Psychonomic Bulletin & Review, 23, 6 (2016), 1713–1743.
   PubMed Web of Science ®Google Scholar
• Cutting, J.E.; DeLong, J.E.; and Nothelfer, C.E. Attention and the evolution of Hollywood film. Psychological Science, 21, 3 (2010), 432–439.
   PubMed Web of Science ®Google Scholar
• de Pinto, M.G.; Polignano, M.; Lops, P.; and Semeraro, G. Emotions understanding model from spoken language using deep neural networks and mel-frequency cepstral coefficients. In 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), 2020, 1–5.
   Google Scholar
• Delgado-Ballester, E.; López-López, I.; and Bernal-Palazón, A. Why do people initiate an online firestorm? The role of sadness, anger, and dislike. International Journal of Electronic Commerce, 25, 3 (2021), 313–337.
   Web of Science ®Google Scholar
• Demmers, J.; Weltevreden, J.W.J.; and van Dolen, W.M. Consumer engagement with brand posts on social media in consecutive stages of the customer journey. International Journal of Electronic Commerce, 24, 1 (2020), 53–77.
   Web of Science ®Google Scholar
• Dobele, A.; Lindgreen, A.; Beverland, M.; Vanhamme, J.; and Van Wijk, R. Why pass on viral messages? Because they connect emotionally. Business Horizons, 50, 4 (2007), 291–304.
   Google Scholar
• Eerola, T. Music and emotion dataset (primary musical cues). Harvard Dataverse, 2016.
   Google Scholar
• Eilam, E. Synchronization: A framework for examining emotional climate in classes. Humanities and Social Sciences Communications, 6, 1 (2019), 144.
   Google Scholar
• Ekman, P. An argument for basic emotions. Cognition & Emotion, 6, 3–4 (1992), 169–200.
   Web of Science ®Google Scholar
• Ekman, P. Facial expressions of emotion: New findings, new questions. Psychological Science, 3, 1 (1992), 34–38.
   Web of Science ®Google Scholar
• Elpers, J.L.W.; Wedel, M.; and Pieters, R.G. Why do consumers stop viewing television commercials? Two experiments on the influence of moment-to-moment entertainment and information value. Journal of Marketing Research, 40, 4 (2003), 437–453.
   Web of Science ®Google Scholar
• Fang, J.; Chen, L.; Wen, C.; and Prybutok, V.R. Co-viewing experience in video websites: The effect of social presence on e-loyalty. International Journal of Electronic Commerce, 22, 3 (2018), 446–476.
   Web of Science ®Google Scholar
• Fredrickson, B.L. What good are positive emotions? Review of General Psychology, 2, 3 (1998), 300–319.
   PubMedGoogle Scholar
• Frommer, D.; and Angelova, K. Chart of the day: Half of YouTube videos get fewer than 500 views. 2009. https://www.businessinsider.com/chart-of-the-day-youtube-videos-by-views-2009-5 (accessed on June 5, 2023).
   Google Scholar
• Gouyon, F.; Pachet, F.; and Delerue, O. On the use of zero-crossing rate for an application of classification of percussive sounds. In Proceedings of the COST G-6 Conference on Digital Audio Effects ( DAFX-00), 5 (2000), 16.
   Google Scholar
• Gui, L.; Yuan, L.; Xu, R.; Liu, B.; Lu, Q.; and Zhou, Y. Emotion cause detection with linguistic construction in Chinese Weibo text. In CCF International Conference on Natural Language Processing and Chinese Computing, 2014, 457–464.
   Google Scholar
• Hennig-Thurau, T.; Wiertz, C.; and Feldhaus, F. Does Twitter matter? The impact of microblogging word of mouth on consumers’ adoption of new movies. Journal of the Academy of Marketing Science, 43, 3 (2015), 375–394.
   Web of Science ®Google Scholar
• Hollebeek, L. Exploring customer brand engagement: definition and themes. Journal of Strategic Marketing, 19, 7 (2011), 555–573.
   Google Scholar
• Hsieh, J.K.; Hsieh, Y.C.; and Tang, Y.C. Exploring the disseminating behaviors of eWOM marketing: persuasion in online video. Electronic Commerce Research, 12, 2 (2012), 201–224.
   Web of Science ®Google Scholar
• Hu, M.; Zhang, M.; and Luo, N. Understanding participation on video sharing communities: The role of self-construal and community interactivity. Computers in Human Behavior, 62, C (2016), 105–115.
   Web of Science ®Google Scholar
• Huan, R.H.; Shu, J.; Bao, S.L.; Liang, R.H.; Chen, P.; and Chi, K.K. Video multimodal emotion recognition based on Bi-GRU and attention fusion. Multimedia Tools and Applications, 80 (2021), 8213–8240.
   Web of Science ®Google Scholar
• Hui, S.K.; Meyvis, T.; and Assael, H. Analyzing moment-to-moment data using a Bayesian functional linear model: Application to TV show pilot testing. Marketing Science, 33, 2 (2014), 222–240.
   Web of Science ®Google Scholar
• Jeon, M. Emotions and affect in human factors and human–computer interaction: Taxonomy, theories, approaches, and methods. In Emotions and Affect in Human Factors and Human-Computer Interaction, 2017, 3–26.
   Google Scholar
• Kensinger, E.A.; and Schacter, D.L. Processing emotional pictures and words: Effects of valence and arousal. Cognitive, Affective, & Behavioral Neuroscience, 6, 2 (2006), 110–126.
   PubMed Web of Science ®Google Scholar
• Kim, C.; and Yang, S.-U. Like, comment, and share on Facebook: How each behavior differs from the other. Public Relations Review, 43, 2 (2017), 441–449.
   Web of Science ®Google Scholar
• Kong, Q.; Cao, Y.; Iqbal, T.; Wang, Y.; Wang, W.; and Plumbley, M.D. PANNs: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28 (2020), 2880–2894.
   Web of Science ®Google Scholar
• Kranzbühler, A.M.; Zerres, A.; Kleijnen, M.H.; and Verlegh, P.W. Beyond valence: A meta-analysis of discrete emotions in firm-customer encounters. Journal of the Academy of Marketing Science, 48, 3 (2020), 478–498.
   Web of Science ®Google Scholar
• Lang, A. The limited capacity model of mediated message processing. Journal of Communication, 50, 1 (2000), 46–70.
   Web of Science ®Google Scholar
• Lazarus, R.S. Cognition and motivation in emotion. American Psychologist, 46, 4 (1991), 352.
   PubMed Web of Science ®Google Scholar
• Lerner, J.S.; Li, Y.; Valdesolo, P.; and Kassam, K.S. Emotion and decision making. Annual Review of Psychology, 66, 1 (2015), 799–823.
   PubMed Web of Science ®Google Scholar
• Lerner, J.S.; Small, D.A.; and Loewenstein, G. Heart strings and purse strings: Carryover effects of emotions on economic decisions. Psychological Science, 15, 5 (2004), 337–341.
   PubMed Web of Science ®Google Scholar
• Lewis, M.; Haviland-Jones, J.M.; and Barrett, L.F. (eds.). Handbook of Emotions. Guilford Press, 2010.
   Google Scholar
• Lin, Y.; Yao, D.; and Chen, X. Happiness begets money: Emotion and engagement in live streaming. Journal of Marketing Research, 58, 3 (2021), 417–438.
   Web of Science ®Google Scholar
• Liu, L.; Dzyabura, D.; and Mizik, N. Visual listening in: Extracting brand image portrayed on social media. Marketing Science, 39, 4 (2020), 669–686.
   Web of Science ®Google Scholar
• Liu, X.; Shi, S.W.; Teixeira, T.; and Wedel, M. Video content marketing: The making of clips. Journal of Marketing, 8 2, 4 (2018), 86–101.
   Google Scholar
• Liu, Y.J.; Yu, M.; Zhao, G.; Song, J.; Ge, Y.; and Shi, Y. Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Transactions on Affective Computing, 9, 4 (2017), 550–562.
   Web of Science ®Google Scholar
• Livingstone, S.R.; and Russo, F.A. The ryerson audio-visual database of emotional speech and song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in north American English. PloS ONE, 13, 5 (2018), e0196391.
   PubMed Web of Science ®Google Scholar
• Lv, G.; Zhang, K.; Wu, L.; Chen, E.; Xu, T.; Liu, Q.; and He, W. Understanding the users and videos by mining a novel danmu dataset. IEEE Transactions on Big Data, 8, 2 (2019), 535–551.
   Google Scholar
• Malik, M.; and Hussain, A. Helpfulness of product reviews as a function of discrete positive and negative emotions. Computers in Human Behavior, 73 (2017), 290–302.
   Web of Science ®Google Scholar
• Morck, R.; Yeung, B.; and Yu, W. The information content of stock markets: Why do emerging markets have synchronous stock price movements? Journal of Financial Economics, 58, 1–2 (2000), 215–260.
   Web of Science ®Google Scholar
• Müller, G.; and Möser, M. (eds.). Handbook of Engineering Acoustics. Springer Science & Business Media, 2012.
   Google Scholar
• Naqvi, N.; Shiv, B.; and Bechara, A. The role of emotion in decision making: A cognitive neuroscience perspective. Current Directions in Psychological Science, 15, 5 (2006), 260–264.
   Web of Science ®Google Scholar
• Nikolinakou, A.; and King, K.W. Viral video ads: Emotional triggers and social media virality. Psychology & Mrketing, 35, 10 (2018), 715–726.
   Web of Science ®Google Scholar
• Oestreicher-Singer, G.; and Zalmanson, L. Content or community? A digital business strategy for content providers in the social age. MIS Quarterly, 37, 2 (2013), 591–616.
   Web of Science ®Google Scholar
• Oliveira, J.S.; Ifie, K.; Sykora, M.; Tsougkou, E.; Castro, V.; and Elayan, S. The effect of emotional positivity of brand-generated social media messages on consumer attention and information sharing. Journal of Business Research, 140, 1 (2022), 49–61.
   Google Scholar
• Pandeya, Y.R.; and Lee, J. Deep learning-based late fusion of multimodal information for emotion classification of music video. Multimedia Tools and Applications, 80 (2021), 2887–2905.
   Web of Science ®Google Scholar
• Pichora-Fuller, M.K.; and Dupuis, K. Toronto emotional speech set (TESS). Scholars Portal Dataverse, 1 (2020).
   Google Scholar
• Plutchik, R. The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American Scientist, 89, 4 (2001), 344–350.
   Web of Science ®Google Scholar
• Quan, C.; and Ren, F. A blog emotion corpus for emotional expression analysis in Chinese. Computer Speech & Language, 24, 4 (2010), 726–749.
   Web of Science ®Google Scholar
• Ren, G.; and Hong, T. Examining the relationship between specific negative emotions and the perceived helpfulness of online reviews. Information Processing & Management, 56, 4 (2019), 1425–1438.
   Web of Science ®Google Scholar
• Rosenbaum-Elliott, R. Strategic Advertising Management. Oxford: Oxford University Press, 2020.
   Google Scholar
• Rui, H.; Liu, Y.; and Whinston, A. Whose and what chatter matters? The effect of tweets on movie sales. Decision Support Systems, 55, 4 (2013), 863–870.
   Web of Science ®Google Scholar
• Russell, J.A. A circumplex model of affect. Journal of Personality and Social Psychology, 39, 6 (1980), 1161–1179.
   Web of Science ®Google Scholar
• Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
   Google Scholar
• Scissors, L.; Burke, M.; and Wengrovitz, S. What’s in a Like? Attitudes and behaviors around receiving Likes on Facebook. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, 2016, 1501–1510.
   Google Scholar
• Seiler, S.; Yao, S.; and Wang, W. Does online word of mouth increase demand? (and how?) Evidence from a natural experiment. Marketing Science, 36, 6 (2017), 838–861.
   Web of Science ®Google Scholar
• She, D.; Yang, J.; Cheng, M.M.; Lai, Y.K.; Rosin, P.L.; and Wang, L. WSCnet: Weakly supervised coupled networks for visual sentiment classification and detection. IEEE Transactions on Multimedia, 22, 5 (2019), 1358–1371.
   Web of Science ®Google Scholar
• Shevlin, R. Guest post: The value of customer engagement. 2017. https://marketingroi.wordpress.com/2007/11/30/the-value-of-customer-engagement (accessed on June 5, 2023).
   Google Scholar
• Song, T.; Huang, J.; Tan, Y.; and Yu, Y. Using user-and marketer-generated content for box office revenue prediction: Differences between microblogging and third-party platforms. Information Systems Research, 30, 1 (2019), 191–203.
   Web of Science ®Google Scholar
• Statistica. Hours of video uploaded to YouTube every minute as of July 2015. 2015. https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute (accessed on June 5, 2023).
   Google Scholar
• Stieglitz, S.; and Dang-Xuan, L. Emotions and information diffusion in social media—sentiment of microblogs and sharing behavior. Journal of Management Information Systems, 29, 4 (2013), 217–248.
   Web of Science ®Google Scholar
• Suri, A.; Huang, B.; and Sénécal, S. This product seems better now: How social media influencers’ opinions impact consumers’ post-failure responses. International Journal of Electronic Commerce, 27, 3 (2023), 297–323.
   Web of Science ®Google Scholar
• Tang, D.; Qin, B.; Liu, T.; and Li, Z. Learning sentence representation for emotion classification on microblogs. In CCF International Conference on Natural Language Processing and Chinese Computing, 2013, 212–223.
   Google Scholar
• Tarpley, T.G.; Fischer, L.M.; Steede, G.M.; Cummins, R.G.; and McCord, A. How much transparency is too much? A moment-to-moment analysis of viewer comfort in response to animal slaughter videos. Journal of Applied Communications, 104, 2 (2020), 6.
   Google Scholar
• Teixeira, T.S.; Wedel, M.; and Pieters, R. Moment-to-moment optimal branding in TV commercials: Preventing avoidance by pulsing. Marketing Science, 29, 5 (2010), 783–804.
   Web of Science ®Google Scholar
• Tellis, G.J.; MacInnis, D.J.; Tirunillai, S.; and Apostolidis, Y. What drives virality (sharing) of online digital content? The critical role of information, emotion, and brand prominence. Journal of Marketing, 83, 4 (2019), 1–20.
   Web of Science ®Google Scholar
• Tomkins, S. Affect Imagery Consciousness: Volume I: The Positive Affects. Springer, 1962.
   Google Scholar
• Tong, L.C.; Acikalin, M.Y.; Genevsky, A.; Shiv, B.; and Knutson, B. Brain activity forecasts video engagement in an internet attention market. Proceedings of the National Academy of Sciences, 117, 12 (2020), 6936–6941.
   Google Scholar
• Weismueller, J.; Harrigan, P.; Coussement, K.; and Tessitore, T. What makes people share political content on social media? The role of emotion, authority and ideology. Computers in Human Behavior, 129, 1 (2022), 107150.
   Google Scholar
• Ye, W.; Heidemann, J.; and Estrin, D. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 12, 3 (2004), 493–506.
   Web of Science ®Google Scholar
• Yin, D.; Bond, S.D.; and Zhang, H. Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS Quarterly, 38, 2 (2014), 539–560.
   Web of Science ®Google Scholar
• Yin, D.; Bond, S.D.; and Zhang, H. Keep your cool or let it out: Nonlinear effects of expressed arousal on perceptions of consumer reviews. Journal of Marketing Research, 54, 3 (2017), 447–463.
   Web of Science ®Google Scholar
• You, Q.; Luo, J.; Jin, H.; and Yang, J. Building a large scale dataset for image emotion recognition: The fine print and the benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 308–314.
   Google Scholar
• Yu, X.; Liu, Y.; Huang, X.; and An, A. Mining online reviews for predicting sales performance: A case study in the movie domain. IEEE Transactions on Knowledge and Data Engineering, 24, 4 (2010), 720–734.
   Web of Science ®Google Scholar
• Yu, Y.; Huang, S.; Liu, Y.; and Tan, Y. Emotions in online content diffusion. Available at SSRN 3724011, 2020.
   Google Scholar
• Yu, Y.; Yang, Y.; Huang, J.; and Tan, Y. Unifying empirical and theoretical perspectives: Emotions in online reviews and product sales. Available at SSRN 3497884, 2019.
   Google Scholar
• Zhang, Q.; Wang, W.; and Chen, Y. Frontiers: In-consumption social listening with moment-to-moment unstructured data: The case of movie appreciation and live comments. Marketing Science, 39, 2 (2020), 285–295.
   Web of Science ®Google Scholar
• Zhao, K.; Hu, Y.; and Lu, Y. (2021). Understanding the role of video quality and emotion in live streaming viewership. In Proceedings of the International Conference on Information Systems, 2021, 1–9.
   Google Scholar