Cross-Task Cognitive Load Classification with Identity Mapping-Based Distributed CNN and Attention-Based RNN Using Gabor Decomposed Data Images

Abstract

The cognitive workload is a key to developing a logical and conscious thinking system. Maintaining an optimum workload improves the performance of an individual. The individuals’ psycho-social factors are responsible for creating significant variability in the performance of a task, which poses a significant challenge in developing a consistent model for the classification of cross-task cognitive workload using physiological signal, Electroencephalogram (EEG). The primary focus of the proposed work is to develop a robust classification model CARNN, by employing the concatenated deep structure of distributed branches of convolutional neural networks with residual blocks through identity mappings, and recurrent neural network with an attention mechanism. EEG data is divided into milliseconds duration overlap segments. The segmented EEG data is converted into images using Gabor decomposition with two spatial frequency scales and four orientations and supplied as input to CARNN. The images are formed by interlacing the respective left and right electrode data to capture the data variations effectively. Efficient feature aggregation with learning of spatial and temporal domain discriminative features through Gabor decomposed data images improve the training of CARNN. CARNN achieves outstanding performance over traditional classifiers; support vector machine, k-nearest neighbor (KNN), ensemble subspace KNN and the pre-trained networks; AlexNet, ResNet18/50, VGG16/19, and Inception-v3. The proposed method results in 94.2%, 92.5%, 95.9%, 92.8%, 94.3% classification accuracy, specificity, sensitivity, precision, and F1-score, respectively. Two visual task levels apart in their complexity are used for cross-task classification of cognitive workload. The proposed method is validated on raw EEG data of 44 participants.

KEYWORDS:

• Attention mechanism
• Cognitive load
• Convolutional neural network (CNN)
• Cross- task
• Electroencephalogram
• Long short- term memory
• Residual block

Acknowledgement

The work was supported by funds from the All India Council for Technical Education (AICTE), New Delhi, under Research Promotion Scheme grant number 8-71/FDC/RPS(POLICY-1)/2019–20.

DISCLOSURE STATEMENT

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by All India Council for Technical Education: [Grant Number 8-71/FDC/RPS(POLICY-1)/2019–20].

Notes on contributors

Trupti Taori

Trupti J Taori received the master’s degree in VLSI in 2010. She is a research scholar at SGGSIET, Nanded since 2019, where she is working on biomedical signal processing. Her research interest includes embedded systems, VLSI and biomedical engineering. Corresponding author. Email: [email protected]

Shankar Gupta

Shankar S Gupta received the master's degree in electronics in 2015. He is a research scholar at SGGSIET, Nanded since 2017, where he is working on biomedical signal processing. His research interest includes biomedical engineering and natural language processing.

Email: [email protected]

Sandesh Bhagat

Sandesh Bhagat received the master's degree in electronics in 2018. He is a research scholar at SGGSIET, Nanded since 2018, where he is working on computer vision based agricultural applications. His research interest includes deep learning and computer vision.

Email: [email protected]

Suhas Gajre

Suhas S Gajre received the PhD degree from Indian Institute of Technology, Delhi, India, in 2007. Presently, he is working as professor in the Department of Electronics and Telecommunication Engineering at SGGSIET, Nanded, India. His research interests include biomedical signal and image processing, and analog and mixed signal VLSI design. Email: [email protected]

Ramchandra Manthalkar

Ramchandra R Manthalkar received the PhD degree from Indian Institute of Technology, Kharagpur, India, in 2003. Presently, he is working as professor in the Department of Electronics and Telecommunication Engineering at SGGSIET, Nanded, India. His research interests include digital signal and image processing, VLSI and computer networks. Email: [email protected]