662
Views
0
CrossRef citations to date
0
Altmetric
Case Report

Correlation analysis between EEG data and facial expressions and sleep behaviors

, , , , &
Article: 2208167 | Received 07 Dec 2022, Accepted 24 Apr 2023, Published online: 13 May 2023

Abstract

Aims

To date, several methods were presented to assess sleep quality, including subjective and objective methods in the clinic. In the present study, the concept-of-sleep quality assessment method was assessed through EEG data, facial expressions, and sleep behaviours and to explore the correlation between EEG data and facial expressions and sleep behavior.

Methods

Sleep EEG data was collected by The Philips Alice 6LDE system, facial expressions were collected from top, left and right angles by an infrared camera, and sleep behavior was recorded by a full-view camera. EEG data were analyzed by EEGLAB, Noldus FaceReader software and Observer XT analyzed facial expression and sleep behavior.

Results

The average power of each band of EEG fluctuated up and down during sleep, the frequency of sleep behaviour was the least in the 3-4 h of sleep, and natural facial expression was the largest in the 3-4 h of sleep. And the correlation analysis results showed that the changes in each band under different channels were correlated with facial expressions (p < 0.05).

Conclusion

This experiment showed the changes in sleep EEG, sleep behaviors and sleep facial expression during sleep, and found the correlation between sleep EEG data and sleep facial expression.

Introduction

Sleep is very important for the human body and it is an active process of the body, which helps to restore the spirit and relieve fatigue [Citation1]. Researchers demonstrated that the lack of sleep affects the brain’s memory ability [Citation2]. Therefore, the study of sleep behaviours can assist scholars to better understand the concept of sleep.

With the increasing pressure of social competition, people’s sleep quality decreases rapidly, and a variety of health challenges or health risks caused by sleep deprivation may gradually occur. Sleep deprivation is associated with a variety of acute and chronic diseases, and it is also associated with the occurrence of adverse events, including cardiovascular events and cerebral and nervous system diseases, resulting in increased morbidity and mortality of cardiovascular diseases [Citation3]. According to the survey, the stress of life and work caused by coronavirus disease 2019 (COVID-19) affected sleep quality maximally in Brazil (64%) and minimally in Germany and Japan (35%). In the United States, 38% of respondents demonstrated that depression and anxiety are the main reasons for low-quality sleep at night. In addition, family stress, job anxiety, and financial stress contribute to sleep disorders [Citation4]. Poor sleep is a relatively common disorder that affects the endocrine, immune and nervous systems, and public health guidelines on the importance of sleep quality and its association with morbidity should be considered [Citation5].

At present, the main sleep quality assessment tools include subjective and objective methods in the clinic. The subjective methods include polysomnography (PSG) [Citation6], electroencephalogram (EEG) [Citation7]. Polysomnography is a basic detection method for sleep disorders recommended by clinical guidelines. Using the changes and rules of PSG during each sleep period, plays a fundamental role in the assessment of sleep quality and identification of sleep disorders. However, polysomnographic monitoring devices are inappropriate for family use due to their expensive price and complicated operation. Secondly, The objective methods including Pittsburgh Sleep Quality Index (PSQI) [Citation8], Epworth Seepiness Scale (ESS)[Citation9], Athens Insomnia Scale (AIS) [Citation10], Berlin Questionnaire (BQ) [Citation11], Mizan Sleep Quality and Sleep Hygiene Index (MiSQuaSHI) [Citation12], Leeds Sleep Assessment Questionnaire -Mizan (LSEQ-M) [Citation12], Daytime Sleepiness Perception Scale 4 (DSPS-4) [Citation13], Sleep Audit [Citation14], chart review and implementation of Children’s Sleep Habits Questionnaire (CSHQ) [Citation15], MySleepScript [Citation16], WatchPAT [Citation17] etc. Although there were a lot of different scale tests, but the scale method is highly subjective and lacks specific, quantifiable, objective, and effective evaluation indicators. Therefore, the study of simple and convenient sleep quality assessment tools is essential for family or individual monitoring of sleep quality, so as to improve sleep quality and promote physical fitness.

The basic movements of the human body are completed under the unified control of the cerebral cortex. Excessive sleep movements and behaviors are reasonably believed to be caused by the excitement or activity of the cerebral cortex. The excitement or activity of the cerebral cortex may also lead to the relative reduction of deep sleep time, thus, there is a certain theoretical basis for the evaluation of sleep quality based on sleep behaviors. To date, studies on sleep behaviors have mainly concentrated on children [Citation18], psychological parameters [Citation19], sleep disorders [Citation20], Parkinson’s disease [Citation21], and other factors. The content of monitoring sleep behaviors, including body movements and muscle tremors, has also been defined in the AASM scoring manual of the American Academy of Sleep Medicine [Citation22]. However, polysomnography should also be used for recording and data collection and analysis through wearing instruments. To sum up, sleep behaviors were more assessed in psychological and Parkinson’s disease patients, the classification of sleep behaviors was more inclined to sleep habits and sleep posture, and the spontaneous body behavior during sleep was not classified and analyzed, which data collection was mainly based on polysomnography, while there was discomfort in normal people wearing instruments during sleep. Moreover, the environmental changes may also easy to cause stress responses, and the measured data may not be consistent with the frequent sleep state. Therefore, collecting data related to sleep behaviors through cameras can better reflect the real state of sleep.

This study aimed to analyze the spontaneous behaviors of the body during sleep, explore the relationship between sleep EEG data, sleep behaviors and facial expressions, see , and analyze the feasibility of developing an objectively reflected sleep quality evaluation system.

Figure 1. Sleep behaviors and facial expressions and EEG combine measure scheme.

Figure 1. Sleep behaviors and facial expressions and EEG combine measure scheme.

Methods

Ethic statement

The present study was approved by the Medical and Health Research Ethics Committee of the Chengdu Medical College (Chengdu, China), and the study was conducted in accordance with the Declaration of Helsinki.

Participant

One Female subject, undergraduate and aged 28-year-old, who met the inclusion criteria was included. The test was conducted after one week of regular sleep. It was attempted to collect 7-h(00:00-07:00) sleep behaviors, facial expressions, and sleep EEG data. The inclusion criteria were as follows: (1) Regular sleep schedule (Pittsburgh Sleep Quality Index score <7); (2) No history of cardiovascular, respiratory, nervous system diseases, mental disorders, or sleep disorders; (3) Non-smokers and non-alcoholics.

EEG detection and analysis

The Philips Alice 6LDE system (Philips, Amsterdam, Netherlands) was used to collect EEG data, and the device meets the basic channel required by the AASM scoring manual of the American Academy of Sleep Medicine standards [Citation22], Namely F3 F4 C3 C4 O1 O2. Therefore, the EEG data collection mainly includes the above channels. About EEG analysis, data were analyzed in a unit of half an hour, we used EEGLAB toolbox [Citation23] of MATLAB was used for data preprocessing. The original EEG data were extracted for preprocessing: first, removed useless electrode EMG and EMG, and filtering range was 1–50, concaved filtering 48–52, and power frequency interference was removed. Then, data were processed in segments for 15 s per segment, after deleting bad segments, the electrical components and other artifacts were removed by independent component analysis (ICA) and re-referenced by the whole-brain average. Afterwards, the relative power of each characteristic wave in each data segment was calculated. Finally, the mean value was calculated as the EEG eigenvalue. After completion, characteristic waves were extracted from sleep data and formed a topographic map. EEG characteristic waves were as follows: delta (0–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–50 Hz).

Facial expressions and sleep behaviors

In the present study, tri-angle infrared cameras were installed to record facial expressions from upside, left side, and right side, according to previous data analysis experience, the camera was placed about 35 cm away from the face to ensure clear and complete recording of facial expressions.; besides, one full-view was considered to record the whole-body movements. The sleep lab was set at 28 degrees [Citation24] to ensure that subjects did not screen their body and facial expressions due to temperature discomfort. All videos were segmented into clips each clip lasted for 30 min. Then, FaceReader software (Noldus, Wageningen, the Netherlands) was used to analyze facial expressions during sleep, allowing the coding of the emotions happy, sad, angry, disgusted, scared, surprised and neutral expressions [Citation25], and the video of facial expressions from three angles was integrated to ensure that there were no missing clips. Meanwhile, Observer XT (Noldus) was utilized for coding the movements that occurred during sleep, We’ve divided sleep behavior into seven behaviors, including face movement, left-hand movement, right-hand movement, body movement, head movement, other movements, and sum number, See Supplementary File 1.

Statistical analysis

Data storage and management were performed using Microsoft 365 software. Continuous variables with normal distribution were presented as mean ± standard deviation; otherwise, they were presented as median (Q25-Q75), and the categorical variables were expressed as frequency. Spearman’s rank correlation analysis was employed to analyze correlation among facial expressions, sleep behaviors, and EEG data.

Results

Sleep EEG data

The average power of five characteristic waves in six channels was obtained by segmenting the EEG data and analyzing them in half an hour (, Supplementary Tables 1–5). The power of alpha, beta, theta, delta, and gamma waves fluctuated during sleep. As only one sample existed, the statistical test results could not be calculated by repeated measures ANOVA, thus, the fluctuation of the average power of each characteristic band could be observed. The average power of the alpha wave increased, then decreased, and increased until the seventh hour of sleep (). The average power of the delta wave showed a negative increase and then a positive increase, and the average power was the highest until the seventh hour of sleep (). The average power of the beta wave was negatively elevated and then decreased, and the average power of the beta wave was the weakest until the seventh hour of sleep (). The average power of the gamma wave showed a negative enhancement at 2–3 and 4–7 h after sleeping. It is noteworthy that there were positive increases in the last 30 min of the first hour and in the first 30 min of the fourth hour (). The average power of theta wave increased negatively during sleep and reached its peak at the seventh hour of sleep ().

Figure 2. The figure shows the alpha topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 2. The figure shows the alpha topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 3. The figure show the delta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 3. The figure show the delta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 4. The figure show the beta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 4. The figure show the beta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 5. The figure shows the gamma topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and a negative increase in blue.

Figure 5. The figure shows the gamma topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and a negative increase in blue.

Figure 6. The figure show the theta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Figure 6. The figure show the theta topographic map of the average power of each wave every 30 min. The darker the color, the stronger the power in the region, with a positive increase in red and negative increase in blue.

Sleep behaviors and facial expressions

The proportion of happy, sad, angry, disgusted, scared, surprised and neutral expressions per half hour was analyzed, and the times of face movement left-hand movement, right-hand movement, body movement, head movement, other movement and sum number were analyzed. It was found that the number of behaviours of both left and right hands reached a peak in the first 1–2 h, and the proportion of surprised expressions was the highest. In the third hour, the frequency of sleep behaviors decreased, and the proportion of natural expression was the largest. At 3–4 h, the frequency of sleep behaviors decreased in a cliff style, and the proportion of natural expression was the highest. At 5–6 h, the number of sleep behaviors decreased and no various facial expressions were recorded, as illustrated in and , Supplementary Table 6.

Figure 7. Trends of sleep behaviours within 7 h of sleep.

Figure 7. Trends of sleep behaviours within 7 h of sleep.

Figure 8. Trends of facial expressions within 7 h of sleep.

Figure 8. Trends of facial expressions within 7 h of sleep.

Correlation analysis

The changes of alpha wave in F3 (r = 0.90, p = 0.00), F4 (r = 0.76, p = 0.00), C3 (r = 0.82, p = 0.00), O1 (r = 0.73, p = 0.00), and O2 (r = 0.73, p = 0.00) channels were positively correlated with the changes of sad expression. The changes of alpha wave in F3 (r = 0.62, p = 0.02), C3 (r = 0.55, p = 0.04), O1 (r = 0.56, p = 0.04), and O2 (r = 0.56, p = 0.04) channels were positively correlated with the changes of scared expression (). The change of beta wave in F3 channel was negatively correlated with neutral expression (r =–0.64, p = 0.01), and positively correlated with sad expression (r = 0.62, p = 0.02) and superior expression (r = 0.71, p = 0.00) (). The change of delta wave in F3 channel was positively correlated with sad expression (r = 0.74, p = 0.02) (). No correlation found between the changes of gamma wave in F3 channel and facial expression (). The change of theta wave in F3 channel was positively correlated with sad expression (r = 0.70, p = 0.01) ().

Table 1. Correlation analysis of alpha wave means power changes with facial expressions and sleep behavior.

Table 2. Correlation analysis of beta wave mean power changes with facial expressions and sleep behavior.

Table 3. Correlation analysis of delta wave mean power changes with facial expressions and sleep behavior.

Table 4. Correlation analysis of gamma wave mean power changes with facial expressions and sleep behavior.

Table 5. Correlation analysis of theta wave mean power changes with facial expressions and sleep behavior.

Discussion

In the present study, the results revealed that the average power intensity of different brain waves changed with the time of falling asleep, and it was correlated with facial expressions. To combine the analysis of facial expressions and sleep behaviors, the average power of each wave band was studied in the order of time of falling asleep. The changing trend of each wave was not consistent with the difference in falling asleep time. The results showed that the alpha wave tended to increase and then declined, while the delta wave tended to be negatively enhanced, the beta wave tended to strengthen and then negatively enhanced, and the gamma wave tended to be negatively enhanced. Theta wave showed positive enhancement and then negative enhancement. The most obvious result is that at 1 h before awakening, all bands showed a positive enhancement trend.

One study previously demonstrated that in the phase of non-rapid eye movement (NREM) sleep, the power of the beta wave in patients with insomnia increased [Citation26], the power of the gamma wave increased [Citation27], and the power of the theta wave decreased [Citation28]. During sleep, the power of the theta wave decreased [Citation29]. Average power is also one of the characteristics that can reflect different stages of sleep. In combination with sleep behaviours and facial expressions, the present study revealed that in the alpha band, the power changes of F3, F4, O1, and O2 were positively correlated with a sad expression. This could be attributed to the poor quality of sleep. Importantly, several studies proved that stimulation under sad emotion could result in the increase of alpha wave energy of the central brain region [Citation30], which could justify the above-mentioned findings.

In the beta band, the power change of F3 channel was negatively correlated with the natural expression, and the sad expression was positively correlated with the surprised expression. It was found that the beta band modulated the neutral and emotional expressions, and the theta band modulated happy and sad expressions [Citation31], which may explain β the reason related to these expressions. Furthermore, in theta band, the change of F3 channel was associated with the sad expression. F3 channel is typically found in the frontal lobe, which is mainly involved in sports and advanced mental functions [Citation32].

Conclusion

This experiment showed the changes in sleep EEG, sleep behaviors and sleep facial expression during sleep, and found the correlation between sleep EEG data and sleep facial expression.

Limitations

Although this study was a case report, the data reflected the trends in different wavebands and behaviours during falling asleep. Due to the data volume, instability and individual differences, no highly strong association was noted between changes in EEG waves and facial expressions or sleep behaviours, indicating the necessity of further large-scale study to validate the results of the present study.

Supplemental material

Supplemental Material

Download MS Word (17.6 KB)

Supplemental Material

Download MS Word (33.2 KB)

Disclosure statement

No potential conflict of interest was reported by the author(s).

References

  • Harris AL, Carmona NE, Moss TG, et al. Testing the contiguity of the sleep and fatigue relationship: a daily diary study. Sleep. 2021;44(5):zsaa252.
  • Kim T, Kim S, Kang J, et al. The common effects of sleep deprivation on human long-term memory and cognitive control processes. Front Neurosci. 2022;16:883848.
  • Liu Wei HK. Research progress on the relationship between sleep deprivation and multisystem related diseases. Medical Recapitulate. 2021;27(23):4696–4701.
  • Tomás CC, Oliveira E, Sousa D, et al. Proceedings of the 3rd IPLeiria’s international health congress: leiria, Portugal. 6–7 May 2016. BMC Health Serv Res. 2016;616(S3):200.
  • Kyprianidou M, Panagiotakos D, Kambanaros M, et al. Quality of sleep in the Cypriot population and its association with multimorbidity: a cross-sectional study. Front Public Health. 2021;9:693332.
  • Jafari B, Mohsenin V. Polysomnography. Clin Chest Med. 2010;31(2):287–297.
  • Dijk DJ. Sleep restriction and emotion, electroencephalography (EEG) and dream recall, and insomnia and punctuality. J Sleep Res. 2012;21(3):233–234.
  • Matsui KA-O, Yoshiike TA-O, Nagao K, et al. Association of subjective quality and quantity of sleep with quality of life among a general population. IJERPH. 2021;18(23):12835.
  • Peng LL, Li JR, Sun JJ[, et al. Reliability and validity of the simplified Chinese version of epworth sleepiness scale. Zhonghua Er Bi Yan Hou Tou Jing Wai Ke Za Zhi. 2011;46(1):44–49.
  • Chen WY, Wu YY, Xu MY, et al. Effect of kangaroo mother care on the psychological stress response and sleep quality of mothers with premature infants in the neonatal intensive care unit. Front Pediatr. 2022;10:879956.
  • Andrechuk CRS, Netzer N, Zancanella E, et al. Cultural adaptation and evaluation of the measurement properties of the Berlin questionnaire for Brazil. Sleep Med. 2019;60:182–187. Aug
  • Manzar MD. Mizan sleep quality and sleep hygiene index MiSQuaSHI: a psychometric investigation. Sleep Sci. 2021;14(2):119–127.
  • Marques DR, Gomes AA, de Azevedo MHP. Profiles of subjective daytime sleepiness through cluster analysis. Psychiatr Q. 2020;91(1):147–163.
  • Gulam S, Xyrichis A, Lee GA. Still too noisy – an audit of sleep quality in trauma and orthopaedic patients. Int Emerg Nurs. 2020;49:100812.
  • Teague MS, Hockenberry M, Kinney JL. Sleep evaluation in the assessment of pediatric attention deficit disorders. J Pediatr Health Care. 2019;33(3):234–241.
  • Lee MJ, Gamaldo A, Peters ME, et al. Assessing sleep concerns in individuals with acquired brain injury: the feasibility of a smartpad sleep tool. J Neuropsychiatry Clin Neurosci. 2021;33(3):225–229.
  • Holmedahl NA-O, Fjeldstad OM, Engan H, et al. Validation of peripheral arterial tonometry as tool for sleep assessment in chronic obstructive pulmonary disease. Sci Rep. 2019;9(1):19392.
  • Servot S, St-Amand A, Rousseau M, et al. Sleep ecology, sleep characteristics and behavior problems in young maltreated children: a scoping review. Child Abuse Negl. 2021;122:105364.
  • Peng A, Tang Y, He S, et al. Association between loneliness, sleep behavior and quality: a propensity-score-matched case-control study. Sleep Med. 2021;86:19–24.
  • Antelmi E, Lippolis M, Biscarini F, et al. REM sleep behavior disorder: mimics and variants. Sleep Med Rev. 2021;60:101515.
  • Sringean J, Stefani A, Marini K, et al. Rapid eye movement sleep behavior disorder and rapid eye movement sleep without atonia are more frequent in advanced versus early Parkinson’s disease. Sleep. 2021;44(9):zsab067.
  • Berry RB, Budhiraja R, Gottlieb DJ, et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American academy of sleep medicine. J Clin Sleep Med. 2012;8(5):597–619.
  • Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9–21.
  • Zheng G, Li K, Wang Y. The effects of High-Temperature weather on human sleep quality and appetite. LID - LID - 270. IJERPH. 2019;16(2):270.
  • Skiendziel T, Rösch AG, Schultheiss OA-O. Assessing the convergent validity between the automated emotion recognition software noldus FaceReader 7 and facial action coding system scoring. PLoS One. 2019;14(10):e0223905.
  • Israel B, Buysse DJ, Krafty RT, et al. Short-term stability of sleep and heart rate variability in good sleepers and patients with insomnia: for some measures, one night is enough. Sleep. 2012;35(9):1285–1291.
  • Perlis ML, Smith MT, Andrews PJ, et al. Beta/gamma EEG activity in patients with primary and secondary insomnia and good sleeper controls. Sleep. 2001;24(1):110–117.
  • Merica H, Blois R, Gaillard JM. Spectral characteristics of sleep EEG in chronic insomnia. Eur J Neurosci. 1998;10(5):1826–1834.
  • Ferri R, Cosentino FI, Manconi M, et al. Increased electroencephalographic high frequencies during the sleep onset period in patients with restless legs syndrome. Sleep. 2014;37(8):1375–1381.
  • Criado JR, Gizer IR, Slutske WS, et al. Event-related oscillations to affective stimuli: heritability, linkage and relationship to externalizing disorders. J Psychiatr Res. 2012;46(2):256–263. Feb
  • Yan T, Dong X, Mu N, et al. Positive classification advantage: tracing the time course based on brain oscillation. Front Hum Neurosci. 2017;11:659.
  • Rosch KS, Mostofsky S. Development of the frontal lobe. Handb Clin Neurol. 2019;163:351–367.