KDViT: COVID-19 diagnosis on CT-scans with knowledge distillation of vision transformer

References

• World Health Organization, Coronavirus disease (COVID-19). 2023; Available from: https://www.who.int/health-topics/coronavirus.
   Google Scholar
• Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16×16 Words: transformers for image recognition at scale; 2020.
   Google Scholar
• Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network; 2015.
   Google Scholar
• Gou J, Yu B, Maybank SJ, et al. Knowledge distillation: a survey. Int J Comput Vis. 2020;129:1789–1819.
   Web of Science ®Google Scholar
• Yadav SS, Jadhav SM. Deep convolutional neural network based medical image classification for disease diagnosis. J Big Data. 2019;6:1–18.
   Google Scholar
• Soares E, Angelov P, Biaso S, et al. SARS-CoV-2 CT-scan dataset: a large dataset of real patients CT scans for SARS-CoV-2 identification. medRxiv; 2020. p. 1–8.
   Google Scholar
• Yang X, He X, Zhao J, et al. COVID-CT-Dataset: a CT Scan Dataset about COVID-19. arXiv preprint arXiv:2003.13865; 2020. p. 1–14.
   Google Scholar
• Ning W, Lei S, Yang J, et al. Open resource of clinical data from patients with pneumonia for the prediction of COVID-19 outcomes via deep learning. Nat Biomed Eng. 2020;4:1197–1207.
   PubMed Web of Science ®Google Scholar
• Alsharman N, Jawarneh I. GoogleNet CNN neural network towards chest CT-coronavirus medical image classification. J Comput Sci. 2020;16:620–625.
   Google Scholar
• Saleh AY, Chin CK, Penshie V, et al. Lung cancer medical images classification using hybrid cnn-svm. Int J Adv Intell Informatics. 2021;7:151–162.
   Google Scholar
• Aytaç UC, Güneş A, Ajlouni N. A novel adaptive momentum method for medical image classification using convolutional neural network. BMC Med Imaging. 2022;22:34.
   PubMed Web of Science ®Google Scholar
• Salehi AW, Khan S, Gupta G, et al. A study of CNN and transfer learning in medical imaging: advantages, challenges, future scope. Sustainability. 2023;15:5930.
   Web of Science ®Google Scholar
• Wang W, Yang Y. Development of convolutional neural network and its application in image classification: a survey. Opt Eng. 2019;58:1–19.
   Web of Science ®Google Scholar
• Roy AM, Bose R, Bhaduri J. A fast accurate fine-grain object detection model based on YOLOv4 deep neural network. Neural Comput Appl. 2022;34:3895–3921.
   Web of Science ®Google Scholar
• Roy AM, Bhaduri J. DenseSPH-YOLOv5: an automated damage detection model based on DenseNet and swin-transformer prediction head-enabled YOLOv5 with attention mechanism. Adv Eng Inform. 2023;56:102007.
   Web of Science ®Google Scholar
• Rajeev R, Samath JA, Karthikeyan NK. An intelligent recurrent neural network with long short-term memory (LSTM) BASED batch normalization for medical image denoising. J Med Syst. 2019;43:234.
   PubMed Web of Science ®Google Scholar
• Zhou L, Liu H, Bae J, et al. Self pre-training with masked autoencoders for medical image classification and segmentation; 2022.
   Google Scholar
• Usman M, Zia T, Tariq A. Analyzing transfer learning of vision transformers for interpreting chest radiography. J Digit Imaging. 2022;35:1445–1462.
   PubMed Web of Science ®Google Scholar
• Almalik F, Yaqub M, Nandakumar K. Self-ensembling vision transformer (SEViT) for robust medical image classification. In: Wang L., Dou Q, Fletcher PT, Speidel S, Li S, editors. Medical image computing and computer assisted intervention - MICCAI 2022. MICCAI 2022. Lecture notes in computer science. Cham: Springer; 2022. vol. 13433. p. 376–386.
   Google Scholar
• Shaker AM, Xiong S. Lung image classification based on long-short term memory recurrent neural network. J Phys Conf Ser. 2023;2467:012007.
   Google Scholar
• Leamons R, Cheng H, Al Shami A. Vision transformers for medical images classifications. In: Arai K, editor. Intelligent systems and applications. IntelliSys 2022. Lecture notes in networks and systems, Cham: Springer; 2023. vol. 544. p. 319–325.
   Google Scholar
• Lee SH, Lee S, Song BC. Vision transformer for small-size datasets; 2021.
   Google Scholar
• Jamil S, Roy AM. An efficient and robust phonocardiography (PCG)-based valvular heart diseases (VHD) detection framework using vision transformer (ViT). Comput Biol Med. 2023;158:106734.
   PubMed Web of Science ®Google Scholar
• Jiang B, Chen S, Wang B, et al. MGLNN: semi-supervised learning via multiple graph cooperative learning neural networks. Neural Netw. 2022;153:204–214.
   PubMed Web of Science ®Google Scholar
• Bi J, Zhu Z, Meng Q. Transformer in computer vision. In 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology, Fuzhou, China, CEI 2021; 2021. p. 178–188.
   Google Scholar
• Han K, Wang Y, Chen H, et al. A survey on vision transformer. IEEE Trans Pattern Anal Mach Intell. 2023;45:87–110.
   PubMed Web of Science ®Google Scholar
• Garcea F, Serra A, Lamberti F, et al. Data augmentation for medical imaging: a systematic literature review. Comput Biol Med. 2023;152:106391.
   PubMed Web of Science ®Google Scholar
• Goceri E. Medical image data augmentation: techniques, comparisons and interpretations. Artif Intell Rev. 2023;56:12561–12605.
   Web of Science ®Google Scholar
• Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need; 2017.
   Google Scholar
• Park N, Kim S. How do vision transformers work? In ICLR 2022 – 10th International Conference on Learning Representations; 2022.
   Google Scholar
• Spolaôr N, Lee HD, Mendes AI, et al. Fine-tuning pre-trained neural networks for medical image classification in small clinical datasets. Multimed Tools Appl. 2023;83:27305–27329.
   Web of Science ®Google Scholar
• Wang Z, Liu Q, Dou Q. Contrastive cross-site learning with redesigned net for COVID-19 CT classification. IEEE J Biomed Health Inform. 2020;24:2806–2813.
   PubMed Web of Science ®Google Scholar
• Aguirre-Alvarez PA, Diaz-Carmona J, Arredondo-Velázquez M. Flexible Systolic Hardware Architecture for Computing a Custom Lightweight CNN in CT Images Processing for Automated COVID-19 Diagnosis. In: Mahmud M, Mendoza-Barrera C, Kaiser MS, Bandyopadhyay A, Ray K, Lugo E, editors. Proceedings of trends in electronics and health informatics. TEHI 2022. Lecture notes in networks and systems. Singapore: Springer; 2023. vol. 675. p. 17–34.
   Google Scholar
• Carvalho ED, Silva RR, Araújo FH, et al. An approach to the classification of COVID-19 based on CT scans using convolutional features and genetic algorithms. Comput Biol Med. 2021;136:104744.
   PubMed Web of Science ®Google Scholar
• Jaiswal A, Gianchandani N, Singh D, et al. Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning. J Biomol Struct Dyn. 2021;39:5682–5689.
   PubMed Web of Science ®Google Scholar
• Loey M, Manogaran G, Khalifa NEM. A deep transfer learning model with classical data augmentation and CGAN to detect COVID-19 from chest CT radiography digital images. Neural Comput Appl. 2020;0123456789:1–13.
   Google Scholar
• Gifani P, Shalbaf A, Vafaeezadeh M. Automated detection of COVID-19 using ensemble of transfer learning with deep convolutional neural network based on CT scans. Int J Comput Assist Radiol Surg. 2021;16:115–123.
   PubMed Web of Science ®Google Scholar
• Ewen N, Khan N. Targeted self supervision for classification on a small covid-19 ct scan dataset. In Proceedings – International Symposium on Biomedical Imaging, Nice, France; 2021 Apr. p. 1481–1485.
   Google Scholar
• Abdar M, Salari S, Qahremani S, et al. UncertaintyFuseNet: robust uncertainty-aware hierarchical feature fusion with ensemble Monte Carlo dropout for COVID-19 detection; 2021. p. 1–16.
   Google Scholar