Managing Emotion In The Workplace: An Empirical Study With Enterprise Instant Messaging

ABSTRACT

Enterprise Instant Messaging (EIM) has become an increasingly important tool for enterprises to operate efficiently and for the employees to communicate smoothly, especially with the recent outbreak of the pandemic. This means that employers and employees are having to adapt to new ways of working, e.g. teleworking or home-based working, and they could experience emotional stress, irritability and anxiety. However, few studies have used sentiment analysis to help employees manage their emotions and past studies mostly applied retrospective sentiment analysis on user-generated content as such as Twitter or the internal enterprise data. In this study we present an Employee Sentiment Analysis and Management System (ESAMS) that continuously monitors the emotions of the employees in real time by analyzing the conversations so the managerial members and the team members can actively manage their emotions or adjust their actions on the spot. As a proof-of-concept, we use Naïve Bayes as our sentiment classifier and achieve an average classification accuracy of 74%. The ESAMS was pilot-tested for one month by 10 participants, who were later interviewed as part of the evaluation. The results show that the ESAMS was helpful in improving team performance and team management.

KEYWORDS:

• Enterprise instant messaging
• emotion management
• machine learning
• sentiment analysis

Introduction

Since the introduction of smart phones, people have become connected with each other than ever. Messaging apps allow asynchronous and synchronous communication between people anywhere and anytime. As the messaging apps gradually replace the conventional SMS and phone services, enterprises are starting to see their potential in connecting their employees for better communication and productivity. Many of these apps are free-to-use, and the competition for a larger market share between them is getting increasingly intense. A survey in 2018 by SimilarWeb¹ showed that mobile apps such as WhatsApp and Facebook Messenger were the dominating messaging apps for telecommunication purposes across the world. These messaging apps also offer Voice over IP capabilities allowing their users to talk on the phone, which makes the apps more appealing to the public.

As a result, Enterprise Instant Messaging (EIM) is becoming a new business territory for the messaging service providers. The idea of EIM is basically an instant messaging service that allows employees to communicate within the enterprise as well as with their business partners, such as vendors, suppliers or even clients. EIM also offers an additional layer of security and privacy because an EIM service can be hosted on the enterprise’s private cloud service. Therefore, many enterprises are now integrating EIM services into their infrastructure to facilitate better communication within the enterprise and to improve productivity (Attaran, Attaran, and Kirkland 2019).

Furthermore, with the recent outbreak of COVID-19 pandemic around the world,² enterprises are increasingly relying on various means of communication tools to coordinate internal and external operations (Attaran, Attaran, and Kirkland 2019). Employers and employees are having to adapt to new ways of working, e.g. teleworking or home-based working, and maintaining communication with colleagues and clients. Various machine learning techniques have been applied to user-generated content to explore how that the users feel toward different topics in terms of open innovation and that they express negative sentiment on culture and business management (Saura, Palacios-Marqués, and Ribeiro-Soriano 2023). Past studies have shown that employees could experience emotional stress, irritability, anxiety or other negative health symptoms (Saura, Ribeiro-Soriano, and Zegarra Saldaña 2022) when working in a non-conventional working environment (Hori and Ohashi 2004; Mann and Holdsworth 2003). Employees’ performance is also affected by their emotions (Tsai, Chen, and Liu 2007).

Motivation and Objectives

Text mining and sentiment analysis on Facebook posts, Twitters and other social media has been studied extensively in the past (Chan et al. 2015; Fire, Puzis, and Elovici 2013; Rahman 2012). Previous studies have also applied sentiment analysis on job or company reviews posted by employees on social platforms to better understand employee engagement and satisfaction (Costa and Veloso 2015; Moniz and de Jong 2014; Zhang and Wang 2020). While analyzing employee reviews can provide firms and enterprises insights into employee satisfaction, there was one major limitation in these studies – that is, they analyzed the employee reviews in a retrospective manner. This means that the management or supervisors would not be able to take actions in a timely manner in response to the employee reviews. While sentiment analysis can help enterprises capture the views of online users regarding their products or services and sometimes their competitors’, few studies have applied sentiment analysis to analyze or help employees manage their emotions in enterprises.

Now with Enterprise Instant Messaging (EIM), we have the opportunity to analyze the content exchanged between the employees and other users. In this study, we leverage the features of EIM, such as promptness, direct communication and interactivity, to analyze the employees’ emotions in real time. We are particularly interested in whether sentiment analysis on EIM can help the employees manage their emotions with two goals: (1) promoting smooth communication and collaboration within teams and (2) improving personal performance at work. Instead of analyzing employee reviews retrospectively, we develop an Enterprise Instant Messaging system within our case study company (it will be described in Section 3.1.1 and referred to as Company A hereafter), which incorporates natural language processing (NLP), text mining and machine learning techniques to help us understand what is being said in the EIM system and to remind the users to manage their emotions. In this study we present an Employee Sentiment Analysis and Management System (ESAMS) that analyzes the real-time conversations between employees. To further understand how effective the ESAMS is in assisting the employees in their daily job and communication, we conduct interviews with several employees, managerial members and staff members from the human resources.

The contributions of our work are as follows: (1) Conversations are monitored in real-time to provide real-time emotion feedback to the employees so they can adjust their emotions accordingly; (2) The ESAMS notifies the managers or supervisors of their team members experiencing negative emotions for a prolong period so they can provide the team members with appropriate support; (3) Senior management and the human resources department can easily identify the key emotion influencers with the emotion network graph so they receive adequate support or counseling before it is too late; (4) Interviews and user evaluation are conducted to show the effectiveness of the ESAMS and that employees’ emotions can be managed in real-time either via personal control or supervisor’s intervention.

Paper Organization

The article is organized as follows. In Section 2 we discuss how emotions have been studied in work places and how emotions can impact productivity followed by reviews on how sentiment analysis has been applied in enterprises. We then introduce our ESAMS in Section 3, which consists of the Enterprise Instant Messaging Platform and the Natural Language Processing Platform. In Section 4, we discuss how we evaluated the ESAMS by inviting 10 users to pilot test it for one month and analyzed the results. Lastly, Section 6 concludes the article.

Background and Related Work

This section sets the context of this study by introducing the background information and some related work. We first discuss what impact personal emotions have on a person’s productivity and team performance at work. We then briefly review some natural language processing techniques that we use to mine and analyze the text content exchanged between colleagues followed by previous work related to our study here.

Emotions and Productivity at Work Places

Lazarus (1991) argued that when an individual encounters a problem, the individual will take certain actions to try to solve the present problem. A such behavior is a problem-oriented response adopted by the individual to deal with the problem. By dealing with the problem in a constructive manner helps alleviate negative feelings experienced by the individual at the time. However, when the individual begins to realize that the present problem cannot be contained or is out of his or her control, he and she will turn to emotion-oriented response to alleviate the negative feelings through complaining or being in denial, which has little effect on solving the problem. Past studies (Spector and Fox 2002; Tsai, Chen, and Liu 2007) revealed that if an employee adopted the emotion-oriented response when facing problems at work, his or her productivity and work performance would be greatly reduced.

A study by Ashforth and Humphrey (1995) suggested that although emotions displayed by members of the same team or organization could create some tension or awkwardness, they do not always have profound effect on the productivity or performance of the team. However, Ashforth and Humphrey pointed out that negative emotions displayed by team leaders or managers have a much more adverse effect on the team. Team members could suffer from mental stress, behavioral change and reduced productivity as the result of a such effect. Therefore, good emotion management for leaders or managers could lead to a better performing team.

Other studies (Locke and Latham 1991; Weiner 1985) have showed that managers with positive emotions and attitudes are more forgiving of mistakes and encourage team cohesion, which leads to better team performance and personal achievement. The members in a cohesive team are more willing to invest in the team and more driven to accomplish the common goals. This in turns makes the team better prepared to take on more challenging assignments.

However, it is inevitable that a team member or a manager will experience or display negative emotions at some point in time when dealing with obstacles or setbacks at work as discussed by Connors et al. in their book (Connors, Smith, and Hickman 1998). It is therefore crucial that an individual, especially the leader or manager, acknowledges negative emotions and takes initiatives to react on the emotions by showing compassion or concerns toward fellow team members.

In summary, it is clear that the positive and negative emotions of individuals and team leaders have different influences on individual performance or team cohesion. It is also possible to help an individual recognize that they are in emotions by showing your concerns, which could further help them deal with problems with problem-oriented response instead of emotion-oriented response. This inspires us to investigate an automated mechanism that proactively monitors the emotions of employees and helps the employees deal with their emotions positively, which in turns improves their work performance.

Sentiment Analysis and Enterprises

Sentiment analysis has been an active research topic for many years. Many studies used Twitter messages to study sentiments expressed by the Twitter users. Earlier studies applied various machine learning techniques to classify Twitter messages into positive and negative sentiments (Go, Bhayani, and Huang 2009). Support Vector Machines were a popular classifier and have proven to be a strong baseline (Barbosa and Feng 2010). As deep learning techniques become popular over the years, several studies employed different techniques, such as word embeddings (Tang et al. 2014) and recurrent neural networks (Dong et al. 2014; İ̇rsoy and Cardie 2014), to classify Twitter messages into sentiments. A study by Kratzwald et al. (2018) applied deep learning and transfer learning techniques to classify text, including Twitter tweets, into various emotions, such as anger, fear, joy, sadness and so on. Another more recent study by Jain, Quamer, and Pamula (2023) introduced a BERT-GAN model that incorporates aspect representation and word sequences and demonstrated good performance in terms of accuracy. Others have used online reviews to study the opinions that users have on a given product or a service (Tang, Qin, and Liu 2015). For example, Stieglitz and Krüger (2011) tried to identify emerging issues regarding a car manufacturer, such as vehicle recall, on Twitter and studied how the users expressed their sentiments. This allows enterprises to leverage social media to understand their customers and to improve their products. Positive and negative emotions expressed by travelers on their blogs have also been studied to allow travel service providers to better understand travelers’ preferences and tourism hotspots (Tse and Zhang 2013). Jain et al. (2022) predict whether the recommendation by air travelers is positive or negative to help other travelers make informed decisions whereas Mishra et al. (2023) propose an decision tree-based automated system that predicts star ratings of restaurant to help with tourism promotions. Sentiment analysis has also been applied to cybercrime prevention (Oseguera et al. 2017; Parapar, Losada, and Barreiro 2014; Potha, Maragoudakis, and Lyras 2016), where the emotions conveyed in online text or through public communities are analyzed to determine whether the individual who left the message has committed suicide, cyberbullying or other sexual crimes. Tourism industry also benefits from the advances in NLP. Classifying videos into emotion categories has also been explored using machine learning techniques (Chen, Chang, and Yeh 2017).

While sentiment analysis can help enterprises capture the views of online users regarding their products or services and sometimes their competitors’, few studies have applied sentiment analysis to analyze or help employees manage their emotions in enterprises. Here we discuss some of the studies that exploited sentiment analysis to understand employees and to make better decisions in enterprises.

Costa and Veloso (2015) collected job reviews written by employees in social platforms to assess key factors such as employee engagement, retention and satisfaction. They used different text representations and applied Support Vector Regression (SVR) and Support Vector Machines (SVM) to predict the outcomes of these key factors. Costa and Veloso reported that word embedding representations yielded better prediction performances than the conventional bag-of-word approach. Zhang and Wang (2020) presented AE-SVM, a hybrid model that combines a variational autoencoder and SVM, which was applied to classify anonymous reviews by the employees of the selected leather goods enterprises into positive and negative emotions. Zhang and Wang also designed a system that incorporated AE-SVM to allow the automation of collecting and analyzing the reviews, but unfortunately did not report the effectiveness of the system. Moniz and de Jong (2014) applied basic sentiment analysis on anonymous employee reviews on their companies and measured the impact of employee satisfaction on the firm earnings. Their findings suggested that the employee sentiment discovered in the reviews discussing the firm outlook contains predictive power for firm earnings.

Gelbard et al. (2018) used sentiment analysis to evaluate human factors, including performance, engagement, leadership, workplace dynamics, organizational developmental support, and learning and knowledge creation. They used the Enron e-mail corpus to study the changes in employee involvement and satisfaction and to assess the human factor patterns and trends. Their results indicated that tracking the corporate e-mail sentiment enables the organization to initiate preventive actions that could promote employees’ morale and to devise new management strategy that elicit further engagement in the new direction. In another study by Aqel and Vadera (2010), sentiment analysis was used to analyze the appraisal forms filled by employees to understand their feelings at work and to decide whether an employee’s background is suitable and related to his or her responsibilities. They argued that a deep integration of sentiment analysis into employee appraisal systems could help both managers and employees achieve organizational goals. A recent study by Chungade and Kharat (2017) investigated how to assess the performance of workers working in a virtual organization where people are geographically dissipated and work for different independent companies. They proposed a performance assessment system that measured the performance of the employees by monitoring their loyalty and role related behaviors based on various performance measures and applied sentiment analysis to identify different opinions about the performance of the workers on which the workers were graded accordingly. In a more recent study, Saura, Ribeiro-Soriano, and Zegarra Saldaña (2022) aimed to identify the challenges and opportunities of remote work through the use of digital technologies and platforms by analyzing user-generated content on Twitter. Latent Dirichlet allocation (LDA) model was applied for topic modeling, which identified 11 topics derived from the tweets. They found that the main challenges for employees included privacy concerns, stress and mental health issues, and the need for upskilling and adapting to new technologies.

Previous studies have primarily focused on either improving the prediction performances of machine learning or deep learning techniques or applying sentiment analysis to reviews or opinions on social platforms. While some studies (Aqel and Vadera 2010; Chungade and Kharat 2017) did exploit sentiment analysis on internal enterprise data to help enterprises with people management, no study has used sentiment analysis in a proactive manner where the emotions of employees or managers are monitored on the spot. Furthermore, in-company training and assistance is recommended for firms and enterprises to help their employees with stress management (Saura, Ribeiro-Soriano, and Zegarra Saldaña 2022). Therefore, we present an enterprise instant messaging system (EIM) named Employee Sentiment Analysis and Management System (ESAMS) where the emotions of the employees are monitored in real time and continuously so that the line managers or the human resources department are notified if an employee is experiencing negative emotions or stress for more than two weeks. Our approach is explained in the next section.

Method

Research Framework

Our goal is to develop an enterprise instant messaging system (EIM) that can help the employees manage their emotions when communicating in the messaging system and provide the employees with appropriate support when experiencing negative emotions for a prolonged period of time. Here we present an Employee Sentiment Analysis and Management System (ESAMS) that analyzes the real-time conversation between an employee and his/her colleagues and provides visual feedback for the emotions of his or her messages. This helps the employee adjust the tones of his or her messages if he or she conveys negative emotions, which could impact the team adversely. The ESAMS also monitors the emotions of employees continuously so that if an employee has been displaying negative emotions for some time, the ESAMS would notify the line managers or the human resources department to provide the suffered employee with appropriate support in order to maintain team cohesion and productivity. The history of emotion is recorded to plot an “emotion network graph” to show the management where the source of negative emotions might come from. This allows the managers and the human resources to speak to the right person to isolate the incident. The concept is illustrated in Figures 1‒3. The ESAMS will be explained in two parts, namely Sentiment Analysis Module and System Architecture.

Figure 1. An employee reports branch status to his boss with negative emotion.

Figure 2. The ESAMS identifies an employee has been showing negative emotions and notifies the HR.

Figure 3. An illustration of the emotion network graph showing user 1’s negative emotions affecting his colleagues’ emotion.

The Case Study Company

Our case study company is a Taiwan-based corporation with around 100 employees that provides enterprise-level messaging services, private cloud-base services, voice-over IP communication and video conferencing tools. They are serving 400 thousand enterprise accounts from various sectors, including manufacturing, medicine, and information technology, and have processed over 8 billion messages. The Employee Sentiment Analysis and Management System (ESAMS) is deployed as part of the company’s internal instant messaging service. We will refer the case study company as Company A hereafter.

Sentiment Analysis Module

The sentiment analysis module consists of three components and they are discussed below.

Dataset Collection and Sentiment Labelling

Each organization comes with its own culture and jargons for work, so some terminology or expressions are organization-specific. We have gained access to Company A’s internal instant messaging system and collected huge volumes of messages for our work. However, these messages are raw text and not labeled with emotions. To automate the labeling process, we used Google Cloud Natural Language API³ to acquire the sentiment scores for the messages. The steps are described below and illustrated in Figure 4.

Figure 4. Data collection and sentiment labelling with Google API.

1. The EIM database contains the original text messages, which are later stored in the sentiment database after the data wrangling process.

2. The NLP Daemon is a background program that retrieves the messages from the sentiment database and prepares the messages in the format used by the Google API.

3. The NLP Daemon sends the prepared messages to the Google API to obtain the sentiment score for each message.

4. Once the NLP Daemon receives the response from the Google API, it saves the sentiment score into the sentiment database for each corresponding message as depicted in Table 1. Note that the original messages were written in Traditional Chinese. The English translations of the original messages in Table 1 are provided for readers’ benefit and are not necessarily accurate word-to-word translations. The “Message in English” column is not present in the system.

5. To create the training dataset for our sentiment analysis models, we need to assign “positive” and “negative” labels to messages. Positive messages are those with sentiment scores ≥ 0.8 and negative messages are those with sentiment scores ≤ −0.6. We then query the sentiment database and extract 10,000 positive and 10,000 negative messages. The reason we define positivity and negativity this way is because we want to avoid using neutral or ambiguous sentiment in our training dataset.

Table 1. Message table in the sentiment database. Note that the “message in English” column is provided only for readability and is not present in the system. English messages are translated by human experts.

ID	Original Message in Traditional Chinese	Message in English (translated by human experts)	Google Sentiment Score	Class Label
M1	已幫 *** 走過一趟產品精神，果然講得零零落落，在郭小妹的分享下，應該很會說故事了，下禮拜 *** 再跟他過基本技術交流即可。	We have taken *** through the product training. *** still needs to work on his speech. We’ll go through the tech specs with him next week.	0.8	Positive
M2	這兩篇很適合給這一代新人最為入值得啟蒙教材。	These two articles would be excellent training materials for the newcomers.	0.9	Positive
M3	沒告知***比較貴，服務水準要求高，我們還是算他們便宜。	They had no idea the service was expensive and of high quality and yet we still gave them a discount.	−0.8	Negative
M4	缺乏一點審美的追求，什麼醜陋的事情都敢做。	Lack of a little aesthetic pursuit, dare to do any ugly things.	−0.7	Negative
M5	把很難很重要的事情一直拖到非解決不可的地步。	Delaying the difficult and important things to the point where they must be solved immediately.	−0.8	Negative

Dataset Pre-Processing

After we have labeled the messages according to their sentiment scores, the messages are pre-processed, which involves Chinese word segmentation, stopword removal, sentiment word dictionary. Figure 5 illustrates the workflow for the pre-processing leading to classifier training, which is described in the next section. Note that the Pos.txt and the Neg.txt represent the “positive” and “negative” labels to messages obtained in Section 3.2.1 (c.f. Figure 4).

Figure 5. Workflow of the pre-processing leading to classifier training.

We used SnowNLP⁴ for the pre-processing work. The details are described below. First word segmentation is applied to both positive and negative messages. The word segmentation of SnowNLP uses a character-based generative model which combines the strengths of word-based generative models and character-based discriminative model (Wang, Zong, and Su 2009). Word segmentation is particularly important for the Chinese language because a word or an expression can often consist multiple characters as seen in Table 2. For example, the message with ID M1 contains a word “product” that is made of two characters “產品” and an expression “零零落落” of four characters that means messy, scattered or sloppy. The word segmentation also isolates punctuations. Next, stopwords and punctuations are removed. The resulting messages are given in Table 3. Then we use the remaining words as the index to construct document vectors. For simplicity, we use term frequency as the term weight in the document vectors as illustrated in Table 4. The document vectors will be the training documents for our sentiment classifier, which is described in the next section.

Table 2. Word segmentation on the original messages. Segmentation is denoted by a forward slash “/”.

ID	Segmented Messages	Class Label
M1	已/幫/***/走/過/一/趟/產品/精神/，/果然/講得/零零落落/，/在/郭/小妹/的/分享/下/，/應/該/很/會/說/故事/了/，/下/禮/拜/***/再/跟/他/過/基/本/技/術/交流/即可/。	Positive
M2	這/兩/篇/很/適/合/給/這/一/代/新/人/最/為入/值得/啟蒙/教材/。	Positive
M3	沒/告知/***/比/較/貴/，/服/務/水/準/要求/高/，/我/們/還/是/算/他/們/便宜/。	Negative
M4	缺/乏/一/點/審美/的/追求/，/什/麼/醜/陋/的/事/情/都/敢/做/。	Negative
M5	把/很/難/很/重要/的/事情/一直/拖/到/非/解/決/不可/的/地步/。	Negative

Table 3. Messages after stopword and punctuations removal.

ID	Segmented Messages	Class Label
M1	已/幫/***/走/過/趟/產品/精神/講得/零零落落/郭/小妹/分享/下/應/該/很/會/說/故事/下/禮/拜/***/再/過/基/技/術/交流/即可	Positive
M2	這/兩/篇/很/適/合/給/這/代/新/人/最/為入/值得/啟蒙/教材	Positive
M3	沒/告知/***/較/貴/服/務/水/準/高/們/還/算/們/便宜	Negative
M4	缺/乏/點/審美/追求/什/麼/醜/陋/事/情/都/敢/做	Negative
M5	很/難/很/事情/拖/非/解/決/地步	Negative

Table 4. Document vectors with term frequency as term weights. F₁, F₂, F₃ … F_n denote there are n dimensions.

ID	F1: 認真	F2: 便宜	F3: 很	…	Fn: 交流	Class Label C
M1	0	0	1	…	1	Positive
M2	0	0	1	…	0	Positive
M3	0	1	0	…	0	Negative
M4	0	0	0	…	0	Negative
M5	0	0	2	…	0	Negative

Sentiment Classifier

For the proposed Employee Sentiment Analysis and Management System (ESAMS), we train a Naïve Bayes classifier on the training dataset obtained in the section 3.2.2. The Naïve Bayes classifier will be used for predicting the sentiment scores of the messages sent by employees, which will be discussed in Section 3.3.2 where the Sentiment Feedback Module is introduced. We use SnowNLP to train our Naïve Bayes classifier. The Naïve Bayes classifier is a probabilistic model that assumes the relationship between all variables or features is independent. The Naïve Bayes classifier tends to perform better when trained on larger collections of data. The Naïve Bayes classifier is explained below. Using the same notation in Table 4, let class label C be a pre-defined category, and F₁, … , F_n be n features (in our case n terms). So, the goal of Naïve Bayes classifier is to calculate the probability of C with the given features F₁, … , F_n using the following equations. In Equation 1(1) $\begin{array}{rl}& \mathrm{P}\left(C|F_1,n,F_n\right)=\frac{P\left(C\right)\times P\left(F_1,n,F_n|C\right)}{P\left(F_1,n,F_n\right)}\\ & \cong P\left(C\right)P\left(F_1|C\right)P\left(F_2|C\right)P\left(F_3|C\right)n\cong P\left(C\right)\prod _{i=1}^{n}P\left(F_i|C\right)\end{array}$(1) , for each message, the Naïve Classifier will produce one probability P(C) for the positive label and one probability for the negative label given that features F₁, … , F_n are present. The label with a greater probability will be assigned to the message as seen in Equation 2(2) $\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{y}\left(f_1,\dots \dots ,f_n\right)=argmaxP\left(C=c\right)\prod _{i=1}^{n}P\left(F_i|C=c\right)$(2) .

(1) $\begin{array}{rl}& \mathrm{P}\left(C|F_1,n,F_n\right)=\frac{P\left(C\right)\times P\left(F_1,n,F_n|C\right)}{P\left(F_1,n,F_n\right)}\\ & \cong P\left(C\right)P\left(F_1|C\right)P\left(F_2|C\right)P\left(F_3|C\right)n\cong P\left(C\right)\prod _{i=1}^{n}P\left(F_i|C\right)\end{array}$(1)

(2) $\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{y}\left(f_1,\dots \dots ,f_n\right)=argmaxP\left(C=c\right)\prod _{i=1}^{n}P\left(F_i|C=c\right)$(2)

System Architecture

This section describes the system architecture and the functions of the proposed Employee Sentiment Analysis and Management System (ESAMS).

System Structure and Functions

For this study, we have implemented the NLP platform as illustrated in Figure 6, which consists of three modules, namely sentiment feedback module, notification and counseling module and daily emotion recording module. The sentiment feedback module analyzes the text messages sent by employees and predicts the sentiment of the sent messages. The notification and counseling module notifies the HR department that an employee might need counseling when he or she is experiencing negative emotions for a prolonged period of time. The daily emotion recording module keeps track of the employees’ emotions for monthly analysis on the individuals and on the team performance. The NLP platform communicates with the Company A’s enterprise instant messaging platform, which consists of the messaging module for text messaging, the API module for interfacing with the NLP platform and the user interface module for displaying vision feedback of the emotions.

Figure 6. System structure of the employee sentiment analysis and management system.

The Sentiment Feedback Module

While it is possible to make predictions on the sentiment of each message sent by an individual, it might not be a useful feature to do so in practice. For example, one may jokingly say “You are in trouble” in a conversation but the sentence would be labeled as negative if it is taken out of context. In order to observe the general emotion of one’s conversation, the sentiment prediction is made on the messages accumulated through a single day instead of on individual messages. That is, if someone starts off the day with negative messages but later recovers with positive messages, the sentiment feedback module would first label his or her conversation as negative and then as neutral or mild positive if the positive messages outscore the negative messages. Each individual will receive a sentiment score on his or her conversation for the day and the score will be recorded in the database on a daily basis for long term analysis. For example, if an individual has had three different conversations, he or she will receive a sentiment score for each of the conversations for the day as depicted in Table 5, which is stored in the sentiment database seen in Figure 6. For the ease of interpretation, the users will be given five levels of sentiment polarity instead of the actual sentiment score. The five levels of sentiment polarity are given below in Table 6.

Table 5. Daily emotion for different conversations.

User	Conversation	Sentiment Score	Date
User 1	User 1 – User 4	0.3	2020/01/22
User 2	User 2 – User 4	0.2	2020/01/22
User 3	User 3 – User 4	0.15	2020/01/22
User 1	User 1 – User 4	0.1	2020/01/23
User 1	User 1 – User 3	0.8	2020/01/23
User 3	User 3 – User 1	0.9	2020/01/23

Table 6. Five levels of sentiment polarity used for sentiment feedback.

Level	Emoticon	Sentiment Score	Polarity
5		0.8 < X ≤ 1.0	Very positive
4		0.6 < X ≤ 0.8	Positive
3		0.4 < X ≤ 0.6	Neutral
2		0.2 < X ≤ 0.4	Negative
1		0.0 < X ≤ 0.2	Very negative

Validation of the Sentiment Feedback Module

To verify the effectiveness of the Naïve Bayes classifier, we carry out a ten-fold cross validation on the dataset we collected in Section 3.2.1. The classification accuracies for the ten-fold cross validation are illustrated in Figure 7, where the x-axis “Value of K” denotes the kth validation. The chart in Figure 7 shows the accuracies obtained across ten validations. The average classification accuracy for the ten-fold cross validation is around 74%, which is an acceptable performance by Company A.

Figure 7. Classification accuracies across the ten-fold cross validation. The x-axis “value of K” denotes the kth validation.

Notification and Counseling Module

One main function of the Employee Sentiment Analysis and Management System (ESAMS) resides in the notification and counseling module. This module constantly monitors the sentiment scores of each employee for a two-week window. If one employee has been showing low sentiment scores for the past two weeks, the module will notify his or her supervisor and the human resources department. The supervisor will then be able to look into the incident and provide the employee with assistance when appropriate. The HR department can also arrange counseling services for the employee if he or she wishes to receive professional support. Choosing two weeks to be the observation window is the result of thorough discussion with the senior managers at Company A. Everyone agreed that the intervention could be too aggressive if the observation window is less than two weeks and that employees might hand in their resignation if the observation window is greater than two weeks. For this study, the NLP platform (cf. Figure 6) accesses the sentiment database every midnight and aggregates the sentiment scores of each individual user recorded in the past two weeks. If an employee’s sentiment scores are very negative (0.0 < X ≤ 0.2) for 80% of the times in the past two weeks, the notification and counseling module will send a notification to the responsible personnel (i.e. supervisor or the HR) through the enterprise instant messaging platform (as illustrated in Figure 6). The observation window and the low sentiment scores for notification can be configured for different situations and needs.

Daily Emotion Recording Module

The daily emotion recording module is responsible for storing the sentiment scores of each conversation and the daily overall sentiment score for each employee. The sentiment feedback module calculates the sentiment scores as described in Section 3.3.2 and passes the scores to the daily emotion recording module, which then stores the data on a daily basis into the sentiment database as illustrated in Figure 6.

System Setup

It is Company A’s requirements that the Employee Sentiment Analysis and Management System (ESAMS) operates with high availability (HA), especially the Enterprise Instant Messaging platform. To ensure high availability of the Enterprise Instant Messaging (EIM) platform, a load balancer is installed and adjusts workload between multiple EIM application servers as depicted in Figure 8. The NLP server is responsible for hosting the NLP platform (cf. Figure 6) and communicates with the EIM application servers to provide the functions described in the previous sections. The NLP server provides the functionality we need for this study and is not required for the operation of Company A, so a load balancer is not installed for the NLP server for the time being. Lastly, the database for the EIM sit behind an internal firewall for additional layer of security. The EIM application servers and the NLP server run on Windows 2012 R2 and the IIS service, and the EIM database runs on Microsoft SQL server.

Figure 8. Setup of the employee sentiment analysis and management system with load balancing and firewalls.

Evaluation and Results

Evaluation Methods

The proposed Employee Sentiment Analysis and Management System (ESAMS) is integrated with Company A’ enterprise instant messaging platform as illustrated in Figures 6 and 8. The entire system is deployed and pilot-tested by the employees at Company A for one month. We have invited personnel from various job functions to understand the ESAMS can help the enterprise from different perspectives. The participating employees include a human resources manager, a product manager, a sales manager, a customer service manager and other employees including developers, UI designers and personal assistants. There are 10 participating employees in total. They have been informed that their messages will be collected and analyzed for research

purposes and signed their consent to the use of data. The ESAMS supports multi-devices. The employees can access the ESAMS on a desktop application or through a web browser as well as on their mobile devices as shown in Figures 9,10 and 11 respectively. We want to understand whether proactive emotion management can help the employees and the enterprises in a productive way. To evaluate how the ESAMS can help the employees with their personal performance at work and managerial functions, we have designed a questionnaire with closed and open questions, which will be used in our interviews with the employees at the end of the one-month pilot test. The questionnaire we used is given in Table 7.

Figure 9. The desktop version of the ESAMS user interface. The emotion for this employee’s conversation is shown in the bottom right corner. In this case the conversation is very positive. The names of the employees and the chat groups are covered for identity protection.

Figure 10. The mobile version of the ESAMS user interface. The emotion for this employee’s conversation is shown in the bottom right corner. In this case the conversation is negative. The names of the employees and the chat groups are covered for identity protection.

Figure 11. The web browser version of the ESAMS user interface. The emotion for this employee’s conversation is shown in the bottom right corner. In this case the conversation is neutral. The names of the employees and the chat groups are covered for identity protection.

Table 7. The questionnaire for the post pilot-test interview.

				1	2	3	4	5
Personal Performance
Q1.	The ESAMS analyzes your emotions in the instant messages and shows your emotions as a reminder. Do you agree that it helps with personal job performance and/or productivity?		Disagree						Agree
People Management
Q2		The ESAMS analyzes your emotions in the instant messages and shows your emotions as a reminder. Do you think it is effective in helping with your emotion management?	Ineffective						Effective
Q3		The ESAMS notifies you when someone has been showing negative emotions. Do you agree that it helps you understand your team members’ capacity better?	Disagree						Agree
Q4		The ESAMS notifies you when someone has been showing negative emotions. Do you agree that it helps improve team performance?	Disagree						Agree
Q5		Do you agree that monitoring and managing employees’ emotions with the ESAMS results in better customer satisfaction?	Disagree						Agree
Q6		Do you agree that the emotion network graph helps you identify the key emotion influencer?	Disagree						Agree
Q7		Do you think the emotion feedback provided by the ESAMS helps you in any other way at work? Please describe.
Q8		Do you have any suggestion to improve the ESAMS in terms of functions and design?

Results

The participating employees used the ESAMS for one month and were interviewed to provide both quantitative and qualitative feedback on their experience. The overall responses to the quantitative questions by the 10 participants are summarized in Table 8. Users 1, 2, 3 and 10 were the managers or the supervisors, and their quantitative responses are summarized in Table 9. The other users were non-managerial team members and their responses are summarized in Table 10. The detailed scores of the quantitative responses can be found in Figure 12. The managers and the supervisors seemed to agree with questions 3 to 6 more than the team members. For questions 1 and 2, it was the other way around. Questions 1 and 2 are concerned with personal performance and personal emotion management. On the other hand, questions 3 to 6 are concerned with non-personal aspects such as team members and team performance. This observation suggests that from the managers’ point of view, the ESAMS helps with team management and team performance but is not necessarily useful for the managers in terms of personal benefit. The non-managerial team members, however, generally agreed that the ESAMS helped them in terms of personal benefit and team work.

Table 8. The overall summary of the responses to the quantitative questions by the 10 participants.

	Average	Standard Deviation	Max	Min
Q1	2.9	0.88	4	1
Q2	3.4	1.07	5	1
Q3	4.2	0.63	5	3
Q4	3.9	0.99	5	2
Q5	3.7	0.95	5	2
Q6	4.1	0.57	5	3

Figure 12. The quantitative responses to Q1 to Q6 of the questionnaire. The x-axis represents the 10 different participants and the y-axis is the score given to the question by the corresponding participan

Table 9. Summary of the responses to the quantitative questions by the managers or supervisors (users 1, 2, 3 and user 10).

	Average	Standard Deviation	Max	Min
Q1	2.5	1.00	3	1
Q2	2.75	1.26	4	1
Q3	4.5	0.58	5	4
Q4	4	1.41	5	2
Q5	3.5	1.29	5	2
Q6	4.25	0.96	5	3

Table 10. Summary of the responses to the quantitative questions by the team members excluding the managers or supervisors.

	Average	Standard Deviation	Max	Min
Q1	3.17	0.75	4	2
Q2	3.83	0.75	5	3
Q3	4.00	0.63	5	3
Q4	3.83	0.75	5	3
Q5	3.83	0.75	5	3
Q6	4.00	0.00	4	4

One aspect that the ESAMS fell short was it did not provide adequate support or assistance when one was showing negative emotion. One of the participants even reported that the negative emotion emoticon made him feel even worse and barely helped him deal with the negative situation he was in. While the majority of the participants reported that they would tone down their wording, chose words more carefully and became more aware of their negative emotion, the emotion feedback was sometime inaccurate and therefore ignored. On the other hand, the general participants’ consensus was that the ESAMS was helpful with the team performance. Eight participants reported that the managers and the supervisors became more sensitive to the team dynamics and more willing to intervene if a team member’s negative emotion became a problem for the team. A number of team members also reported that they took initiative to assist another member who had been in negative emotions for a while. The managers and the supervisors praised the ESAMS’s notification feature, which gave them the opportunity to understand how their team performed and what could lead to the resignation of a team member. With the ESAMS, the managers were able to move a team member to a different job function before the team fell apart because of his or her negative emotions or attitudes.

The qualitative feedback received in the interviews was summarized below.

Table

		Avg.	Low	High
Q1.	The ESAMS analyzes your emotions in the instant messages and shows your emotions as a reminder. Do you agree that it helps with personal job performance and/or productivity?	2.90	1	4
Eight of the ten participants reported that they were aware of their conversation being labelled with negative emotion but they did not take any further actions and sometimes they believed that the prediction was incorrect. One of the participants even reported that the negative emotion emoticon made him feel even worse and barely helped him deal with the negative situation he was in. One participant reported that the emotion was inconspicuous in the bottom-right corner of the chat window and that he would not notice it when very occupied with work. By the time he finished work and saw the negative emoticon, his negative feels were already gone and therefore it did not help much.
		Avg.	Low	High
Q2	The ESAMS analyzes your emotions in the instant messages and shows your emotions as a reminder. Do you think it is effective in helping with your emotion management?	3.40	1	5
When the negative emoticon showed up, more than half of the participants reported that they would tone down their wording, chose words more carefully and became more aware of their negative emotion. However, two participants commented that the emotion prediction based on text was not always accurate. For example, responses such as “ok” or “thanks” sometimes are meaningless and often used when someone is busy with work at the time.
		Avg.	Low	High
Q3	The ESAMS notifies you when someone has been showing negative emotions. Do you agree that it helps you understand your team members’ capacity better?	4.20	3	5
Eight of the ten participants agreed that the managers or the supervisors became more aware of the emotional state an individual was in and more sensitive to change in team dynamics. When notified of someone being in negative emotions for two weeks, the managers and the supervisors were more willing to investigate the situation and later provided support according to the nature of the situation.
		Avg.	Low	High
Q4	The ESAMS notifies you when someone has been showing negative emotions. Do you agree that it helps improve team performance?	3.90	2	5
Seven participants reported that when they learned someone had been in negative emotions for a while, they took problem-solving adjustment to support that individual. This prevented a personal-level problem from turning into a team-level problem. A few participants took a less positive view on this question and commented that although external support helps from time to time, the team still need to move on regardless of whether this person is willing to take suggestions or make the adjustment.
		Avg.	Low	High
Q5	Do you agree that monitoring and managing employees’ emotions with the ESAMS results in better customer satisfaction?	3.70	2	5
Six participants reported that for those who need to communicate with internal and external stakeholders or clients, the reminder of negative emotion helped them adjust the tone of the conversations before it got too emotional. They reported that the clients seemed happier when their conversations were of less negative tones and that their supervisors provided assistance in troubleshooting the client’s problems because their supervisors were made aware of their negative emotions through the ESAMS’s notification. However, one participant commented that he speaks to the colleagues on the ESAMS to unload his frustration with the clients, which helps him release some of the stress at work. But the ESAMS keeps telling him his conversation was too negative and now he cannot use the instant messaging platform to share his feelings with his colleagues, which does not help with deal with clients’ requests.
		Avg.	Low	High
Q6	Do you agree that the emotion network graph helps you identify the key emotion influencer?	4.10	3	5
Most participants agreed that it is easier to identify the source or the cause of the negative atmosphere within the team with the emotion network graph. This feature is particularly useful for the supervisor or the HR to visualize the team dynamics and to intervene at the right time before the entire team become emotional and unstable.
Q7	Do you think the emotion feedback provided by the ESAMS helps you in any other way at work? Please describe.
Most participants felt that after using the ESAMS for a while, they became more conscious of their word choice for their messages and tried to show less negativity towards their colleagues.
Q8	Do you have any suggestion to improve the ESAMS in terms of functions and design?
The emotion scores are reset at the beginning of each day, and this leads to fluctuating emotion feedback when the scores are calculated on very few conversations. So it would be more helpful if the emotion feedback is provided after enough conversations have been exchanged. Sometime programmers copy and paste codes in messages and these codes should be ignored for emotion feedback. Users use emoticons in the messages to express emotions but these emoticons are not considered in the emotion feedback. The emoticons of the emotion feedback should be customizable to make the ESAMS more personal. It would be more helpful if the ESAMS gives encouraging or positive messages instead of emoticons as feedback. Also, it might also help if there is a reward mechanism when the user became positive because of the emotion feedback. This should further encourage the users to use the ESAMS. From the customer service’s point of view, it would be more helpful if the emotion feedback is given on individual sentences rather than on the entire conversation. For examples, negative sentences are highlighted so the customer service agents know when and where they should correct their messages when speaking to the client.

The participants felt the ESAMS was less effective in personal performance and personal emotion management. This could be because the current ESAMS provides little customizable features and is not tailored to “personal vocabularies”. Everyone has different expressions and is accustomed to a given set of vocabularies. Some users like to make sarcastic comments while others express their feelings blatantly, and yet this user behavior was not considered in the sentiment feedback module. Furthermore, the ESAMS was evaluated for only one month, which might not be long enough for the participants to realize improved personal performance. The managers and the supervisors even tend to disagree that the ESAMS help improve personal performance. This was because it was the managers’ job to make sure their teams operate smoothly and it was not in their expectation that the ESAMS should help with their personal performance. It is our goal to make the ESAMS more personal to the users in the future so the emotion feedback would be more relevant to the users. The responses to Question 8 will be taken into account in designing the ESAMS version 2.

Discussion

The implementation of the Employee Sentiment Analysis and Management System (ESAMS) within the context of enterprise instant messaging (EIM) marks a significant advancement in the realm of employee well-being and organizational management. Considering the challenges presented by the COVID-19 pandemic, remote working and teleworking have become essential modes of operation for numerous enterprises. This transformation in the workplace has brought about emotional stress, irritability, and anxiety among employees. It is evident that addressing the emotional well-being of employees is not merely a matter of compassionate concern but also vital for maintaining organizational productivity and overall effectiveness.

One notable aspect of this study is the real-time monitoring of employee emotions through ESAMS, which stands in contrast to previous studies that have mainly employed retrospective analysis. Real-time sentiment analysis offers a dynamic perspective on employee emotions, enabling immediate action by both managerial members and team members. Such an approach aligns with the need for proactive management strategies to support employees facing emotional challenges in the remote work environment.

The utilization of the Naïve Bayes classifier in sentiment analysis achieved an average classification accuracy of 74%. While this accuracy level demonstrates the system’s effectiveness, it also signifies the scope for improvement in sentiment analysis algorithms. Future research might explore the integration of more advanced natural language processing and machine learning techniques to enhance the accuracy and granularity of emotion detection.

Additionally, the pilot testing of ESAMS over one month by 10 participants, followed by interviews for evaluation, yielded promising results. The system was found to be beneficial in enhancing team performance and management. This underscores the practical utility of ESAMS in addressing emotional challenges within the enterprise setting.

Lastly, the ESAMS offers a number of key advantages over existing solutions to sentiment analysis:

1. Customization for Enterprise Context: ESAMS was designed to cater to the unique requirements of enterprises, where the dynamics of communication, the nuances of language, and the importance of real-time feedback are distinct. While Google API provides sentiment analysis, it may not be optimized for the specific language patterns and emotions relevant to workplace conversations.

2. Real-Time Monitoring: Unlike traditional sentiment analysis, ESAMS provides real-time monitoring and feedback to employees, allowing them to address emotional challenges as they arise.

3. Privacy and Data Control: Enterprises often have stringent data privacy and security concerns. By using ESAMS, organizations have more control over their data, which can be a significant advantage when dealing with sensitive workplace communications.

4. Proof of Concept: Our research serves as a proof of concept to demonstrate the feasibility of such a system within the EIM context. While the Google API is functional, it may not be optimized for the same use case, and ESAMS represents a step toward creating specialized tools for this domain.

Conclusion

Enterprise Instant Messaging (EIM) has become an increasingly important tool for enterprises to operate efficiently and for the employees to communicate smoothly, especially with the recent outbreak of the pandemic. Past studies showed that employees could experience emotional stress, irritability or anxiety with teleworking. Also, previous studies only applied post sentiment analysis on the internal data. In this study we present an Employee Sentiment Analysis and Management System (ESAMS) that continuously monitors the emotions of the employees in real time by analyzing the conversations so the managers and the team members can manage their emotions or adjust their actions on the go. Another contribution is that, our ESAMS notifies the managers or the HR department when their team members require attention or support if they have been experiencing negative emotions for two weeks, which should reduce the turnover rate according to the participants.

We invited 10 employees, four of which were managers, to use the ESAMS for one month. The participants were then interviewed to evaluate the effectiveness of ESAMS in emotion management and people management. Our evaluation revealed that the participants became more aware of their emotions and took appropriate actions to adjust their emotions with careful choice of words in the messages. The results showed that the participants felt the ESAMS was particularly helpful with team performance and team management. The managers and the supervisors became more aware of the team dynamics and more willing to intervene when team members experienced negative emotions. The participants felt that improvement can be made to make the ESAMS more personal to the users so the emotion feedback would be more relevant to their job functions. We plan to incorporate the participants’ feedback in the next version of the ESAMS.

Theoretical Implications

From a theoretical perspective, the introduction of ESAMS contributes to the burgeoning field of employee sentiment analysis and management. While previous research has primarily relied on post-sentiment analysis of internal enterprise data, ESAMS takes a proactive approach by monitoring emotions in real time. This paradigm shift has theoretical implications for understanding the dynamics of emotional well-being in teleworking and remote work settings.

The utilization of the Naïve Bayes classifier as a proof-of-concept opens avenues for further exploration in sentiment analysis. Future research could delve into more sophisticated machine learning and deep learning techniques, enhancing our understanding of employee emotions and facilitating the development of more precise sentiment analysis models.

Practical Implications

The practical implications of ESAMS are profound. As organizations navigate the challenges presented by the COVID-19 pandemic and the shift toward remote work, ESAMS offers a valuable tool for promoting employee well-being and optimizing team performance. This system empowers both managerial members and team members to address emotional stress, irritability, and anxiety in real time, fostering a more harmonious and productive work environment.

For organizations, the implementation of ESAMS can lead to improved team performance and management, ultimately enhancing overall productivity. Moreover, it can serve as a foundation for the development of support mechanisms and interventions aimed at improving employee emotional well-being in the context of teleworking or remote work. In practice, ESAMS can serve as an early warning system, and its insights should be interpreted by human judgment. Employees and managers can use ESAMS data as a starting point for conversations, discussions, and interventions, rather than making critical decisions solely based on automated sentiment analysis. While the ESAMS accuracy score may not be perfect, it represents a significant step forward in harnessing technology to assist employees in managing emotions in the workplace.

In summary, ESAMS stands as an innovative solution to the emotional challenges faced by employees in the contemporary enterprise landscape. Its theoretical and practical implications underscore its potential as a transformative tool for the management of employee sentiment and the optimization of team dynamics.

Research Limitations and Future Work

The research limitations of this study were as follows: (1) some of our participants were junior members on the team and they seemed to be less straightforward with their responses because they were concerned that their feedback will be used in their performance evaluation; (2) the pilot test lasted only one month and therefore the effectiveness of the ESAMS on personal performance was not proven; (3) currently the ESAMS was built upon SnowNLP and therefore performance of ESAMS was coupled with SnowNLP. More sophisticated methods will be explored in the future.

We have demonstrated the effectiveness of ESAMS as a proof-of-concept on real-time emotion management for employees. In the future we would like to improve the sentiment classification accuracy and to make the ESAMS more personal to the users in the future so the emotion feedback would be more relevant to the users. Another feature is to incorporate language models into the ESAMS so that it generates encouraging or positive messages to boost employees’ morale.

Acknowledgements

We would like to thank the participants for taking their time to test the system and their insightful feedback. We are also grateful to Company A for allowing us to carry out this empirical study within their infrastructure.

Data Availability Statement

The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research supporting data is not available.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1. Mobile Messaging App Map of the World: https://www.similarweb.com/blog/mobile-messaging-app-map-january-2019

2. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports

3. https://cloud.google.com/natural-language

4. https://github.com/isnowfy/snownlp