A hybrid brain-computer interface using motor imagery and SSVEP Based on convolutional neural network

References

• Yin W, Liang Z, Zhang J, et al. Partial least square regression via Three-Factor SVD-Type manifold optimization for EEG decoding. In: Chinese conference on pattern recognition and computer vision (PRCV). Berlin: Springer; 2022, pp. 778–787.
   Google Scholar
• Sadiq MT, Yu X, Yuan Z, et al. Motor imagery EEG signals decoding by multivariate empirical wavelet transform-based framework for robust brain–computer interfaces. IEEE Access. 2019;7:171431–171451. doi: 10.1109/ACCESS.2019.2956018.
   Web of Science ®Google Scholar
• Amin SU, Muhammad G, Abdul W, et al. Multi-CNN feature fusion for efficient EEG classification. In: 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). 2020, pp. 1–6. doi: 10.1109/ICMEW46912.2020.9106021.
   Google Scholar
• Kwon OY, Lee MH, Guan C, et al. Subject-independent brain–computer interfaces based on deep convolutional neural networks. IEEE Trans Neural Netw Learn Syst. 2020;31(10):3839–3852. doi: 10.1109/TNNLS.2019.2946869.
   PubMed Web of Science ®Google Scholar
• Coyle D, Garcia J, Satti AR, et al. EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game in 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB). 2011, p. 1–7.
   Google Scholar
• LaFleur K, Cassady K, Doud A, et al. Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface. J Neural Eng. 2013;10(4):046003. Available at doi: 10.1088/1741-2560/10/4/046003.
   PubMed Web of Science ®Google Scholar
• Akram F, Han SM, Kim TS. An efficient word typing p300-bci system using a modified t9 interface and random Forest classifier. Comput Biol Med. 2015;56:30–36.Available at https://www.sciencedirect.com/science/article/pii/S0010482514002959. doi: 10.1016/j.compbiomed.2014.10.021.
   PubMed Web of Science ®Google Scholar
• Jin J, Sellers EW, Zhou S, et al. A p300 brain–computer interface based on a modification of the mismatch negativity paradigm. Int J Neural Syst. 2015;25(3):1550011. Available at doi: 10.1142/S0129065715500112.
   PubMed Web of Science ®Google Scholar
• Ravi A, Beni NH, Manuel J, et al. Comparing user-dependent and user-independent training of CNN for SSVEP BCI. J Neural Eng. 2020;17(2):026028. Available at doi: 10.1088/1741-2552/ab6a67.
   PubMed Web of Science ®Google Scholar
• Nakanishi M, Wang Y, Wang YT, et al. A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials. PLoS One. 2015;10(10):e0140703. Available at doi: 10.1371/journal.pone.0140703.
   PubMed Web of Science ®Google Scholar
• Decety J. The neurophysiological basis of motor imagery. Behav Brain Res. 1996;77(1–2):45–52. doi: 10.1016/0166-4328(95)00225-1.
   PubMed Web of Science ®Google Scholar
• Decety J, Ingvar DH. Brain structures participating in mental simulation of motor behavior: a neuropsychological interpretation. Acta Psychol (Amst). 1990;73(1):13–34. Available at https://www.sciencedirect.com/science/article/pii/000169189090056L. doi: 10.1016/0001-6918(90)90056-l.
   PubMed Web of Science ®Google Scholar
• Yu Y, Liu Y, Jiang J, et al. An asynchronous control paradigm based on sequential motor imagery and its application in wheelchair navigation. IEEE Trans Neural Syst Rehabil Eng. 2018;26(12):2367–2375. doi: 10.1109/TNSRE.2018.2881215.
   PubMed Web of Science ®Google Scholar
• Wang X, Lu H, Shen X, et al. Prosthetic control system based on motor imagery. Comput Methods Biomech Biomed Engin. 2022;25(7):764–771. pMID: 34533381. doi: 10.1080/10255842.2021.1977800.
   PubMed Web of Science ®Google Scholar
• Chen X, Wang Y, Nakanishi M, et al. High-speed spelling with a noninvasive brain–computer interface. Proc Natl Acad Sci USA. 2015;112(44):E6058–E6067. Available at doi: 10.1073/pnas.1508080112.
   PubMed Web of Science ®Google Scholar
• Wen D, Liang B, Zhou Y, et al. The current research of combining multi-modal brain-computer interfaces with virtual reality. IEEE J Biomed Health Inform. 2021;25(9):3278–3287. doi: 10.1109/JBHI.2020.3047836.
   PubMed Web of Science ®Google Scholar
• Allison BZ, Brunner C, Kaiser V, et al. Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng. 2010;7(2):26007. Available at doi: 10.1088/1741-2560/7/2/026007.
   PubMed Web of Science ®Google Scholar
• Brunner C, Allison BZ, Krusienski DJ, et al. Improved signal processing approaches in an offline simulation of a hybrid brain–computer interface. J Neurosci Methods. 2010;188(1):165–173. Available at https://www.sciencedirect.com/science/article/pii/S0165027010000786. doi: 10.1016/j.jneumeth.2010.02.002.
   PubMed Web of Science ®Google Scholar
• Yu T, Xiao J, Wang F, et al. Enhanced motor imagery training using a hybrid BCI with feedback. IEEE Trans Biomed Eng. 2015;62(7):1706–1717. doi: 10.1109/TBME.2015.2402283.
   PubMed Web of Science ®Google Scholar
• Ko LW, Ranga SSK, Komarov O, et al. Development of single-channel hybrid bci system using motor imagery and ssvep. J Healthc Eng. 2017;2017:3789386–3789387. doi: 10.1155/2017/3789386.
   PubMed Web of Science ®Google Scholar
• Chi X, Wan C, Wang C, et al. A novel hybrid brain-computer interface combining motor imagery and intermodulation steady-state visual evoked potential. IEEE Trans Neural Syst Rehabil Eng. 2022;30:1525–1535. doi: 10.1109/TNSRE.2022.3179971.
   PubMed Web of Science ®Google Scholar
• Yu J, Li C, Lou K, et al. Embedding decomposition for artifacts removal in eeg signals. J Neural Eng. 2022;19(2):026052. doi: 10.1088/1741-2552/ac63eb.
   Web of Science ®Google Scholar
• Qu Y, Jian X, Che W, et al. Transfer learning to decode brain states reflecting the relationship between cognitive tasks. In: Human brain and artificial intelligence. Singapore: Springer Nature; 2023; pp. 110–122.
   Google Scholar
• Schirrmeister RT, Springenberg JT, Fiederer LDJ, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp. 2017;38(11):5391–5420. Available at doi: 10.1002/hbm.23730.
   PubMed Web of Science ®Google Scholar
• Zheng WL, Dong BN, Lu BL. Multimodal emotion recognition using EEG and eye tracking data in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2014, p. 5040–5043.
   Google Scholar
• Xu X, Tao Z, Ming W, et al. Intelligent monitoring and diagnostics using a novel integrated model based on deep learning and multi-sensor feature fusion. Measurement. 2020;165:108086. doi: 10.1016/j.measurement.2020.108086.
   Web of Science ®Google Scholar
• Amin SU, Alsulaiman M, Muhammad G, et al. Deep learning for EEG motor imagery classification based on multi-layer CNNS feature fusion. Future Gener Comput Syst. 2019;101:542–554. doi: 10.1016/j.future.2019.06.027.
   Web of Science ®Google Scholar
• Lee MH, Kwon OY, Kim YJ, et al. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy. GigaScience. 2019;8(5). Available at doi: 10.1093/gigascience/giz002,giz002.
   Web of Science ®Google Scholar
• Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift, in proceedings of the 32nd international conference on machine learning. In: Bach F, Blei D, editors. Proceedings of machine learning research. Vol. 37, Lille, France. PMLR; 2015; pp. 448–456. Available at https://proceedings.mlr.press/v37/ioffe15.html.
   Google Scholar
• Wang S, Manning C. Fast dropout training. In International conference on machine learning. Atlanta, GA: PMLR; 2013; pp. 118–126.
   Google Scholar
• Gal Y, Hron J, Kendall A. Concrete dropout. Advances in Neural Information Processing Systems 30; 2017.
   Google Scholar
• Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting. J Machine Learning Res. 2014;15:1929–1958.
   Web of Science ®Google Scholar
• Kingma DP, Ba J. Adam: a method for stochastic optimization. 2014. Available at https://arxiv.org/abs/1412.6980.
   Google Scholar
• Tang X, Li W, Li X, et al. Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network. Expert Syst Appl. 2020;149:113285. doi: 10.1016/j.eswa.2020.113285.
   Web of Science ®Google Scholar
• Agarap AF. Deep learning using rectified linear units (relu). arXiv Preprint arXiv:1803.08375. 2018.
   Google Scholar
• Clevert DA, Unterthiner T, Hochreiter S. Fast and accurate deep network learning by exponential linear units (elus). arXiv Preprint arXiv:1511.07289. 2015.
   Google Scholar
• Zhang X, Zou Y, Shi W. Dilated convolution neural network with LeakyReLU for environmental sound classification. In: 2017 22nd international conference on digital signal processing (DSP). IEEE; 2017, pp. 1–5. doi: 10.1109/ICDSP.2017.8096153.
   Google Scholar
• Nwankpa C, Ijomah W, Gachagan A, et al. Activation functions: comparison of trends in practice and research for deep learning. arXiv Preprint arXiv:1811.03378. 2018.
   Google Scholar
• Pedamonti D. Comparison of non-linear activation functions for deep neural networks on MNIST classification task. arXiv Preprint arXiv:1804.02763. 2018.
   Google Scholar
• Chiang KJ, Wei CS, Nakanishi M, et al. Boosting template-based SSVEP decoding by cross-domain transfer learning. J Neural Eng. 2021;18(1):016002. Available at doi: 10.1088/1741-2552/abcb6e.
   Web of Science ®Google Scholar