Face manipulated deepfake generation and recognition approaches: a survey

References

• Carrington, d.: (2020 website: Mylio.com) how many photos in 2020? detailed report here (2020); 2020. Available from: https://blog.mylio.com/how-many-photos-will-be-taken-in-2020///.
   Google Scholar
• Cole s (2017) ai-assisted fake porn is here and we’re all fucked; 2017. Available from: https://www.vice.com/enus/article/gydydm/gal-gadot-fake-ai-porn//.
   Google Scholar
• Generator and discriminator; 2021. Available from: https://www.simplilearn.com/tutorials/deep-learning-tutorial/generative-adversarial-networks-gans////.
   Google Scholar
• Kingra S, Aggarwal N, Kaur N. Emergence of deepfakes and video tampering detection approaches: a survey. Multimedia Tools Appl. 2023;82(7):10165–10209. doi: 10.1007/s11042-022-13100-x
   Web of Science ®Google Scholar
• Sarah and acosta doctored video; 2018. Available from: https://www.washingtonpost.com/technology/2018/11/08/white-house-shares-doctored-video-support-punishment-journalist-jim-acosta////.
   Google Scholar
• Obama B; 2017. Available from: https://en.wikipedia.org/wiki/Deepfake#cite_note-118//.
   Google Scholar
• Markzuckerbergdeepfakeexample; 2018. Available from: https://timesofindia.indiatimes.com/gadgets-news/a-fake-video-of-facebook-ceo-mark-zuckerberg-gets-posted-on-instagram-heres-how-the-company-is-responding-to-it/articleshow/69754335.cms////.
   Google Scholar
• Deepfake; 2018. Available from: https://github.com/Deepfakes//.
   Google Scholar
• Face app; 2016. Available from: http://www.faceapp.com//.
   Google Scholar
• Reface; 2020. https://play.google.com/store/apps/details?id=video.reface.app hl=enINgl=US//.
   Google Scholar
• Face swap; 2019. Available from: https://faceswap.dev/.
   Google Scholar
• Gan; 2021. Available from: www.analyticsvidhya.com/blog/2021/05/stylegan-explained-in-less-than-five-minutes/#h2_9//.
   Google Scholar
• Zao; 2019. Available from: https://www.zaoapp.net/.
   Google Scholar
• Nirkin Y, Wolf L, Keller Y, et al. Deepfake detection based on discrepancies between faces and their context. IEEE Transactions on Pattern Analysis and Machine Intelligence; Online. 2021. p. 6111–6121.
   Google Scholar
• He Z, Zuo W, Kan M, et al. Attgan: facial attribute editing by only changing what you want. IEEE Trans Image Process. 2019;28(11):5464–5478. doi: 10.1109/TIP.2019.2916751
   PubMed Web of Science ®Google Scholar
• Tachibana H, Uenoyama K, Aihara S Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Calgary, AB, Canada. IEEE; 2018. p. 4784–4788.
   Google Scholar
• Thies J, Zollhofer M, Stamminger M, et al. Face2face: real-time face capture and reenactment of rgb videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Las Vegas, NV, USA. IEEE; 2016. p. 2387–2395.
   Google Scholar
• Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Adv Neural Inf Process Syst. 2014;27:2672–2680.
   Google Scholar
• Kingma DP, Welling M. Auto-encoding variational bayes. arXiv Preprint arXiv. 2013;{abs/1312.6114}.
   Google Scholar
• Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv Preprint arXiv. 2015;{abs/1511.06434}.
   Google Scholar
• Karras T, Laine S, Aila T A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Long Beach, CA, USA. IEEE; 2019. p. 4401–4410.
   Google Scholar
• Choi Y, Choi M, Kim M, et al. Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Salt Lake City, UT, USA. IEEE; 2018. p. 8789–8797.
   Google Scholar
• Karras T, Aila T, Laine S, et al. Progressive growing of gans for improved quality, stability, and variation. arXiv Preprint arXiv. 2017;{abs/1710.10196}.
   Google Scholar
• Karras T, Laine S, Aittala M, et al. Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Seattle, WA, USA. IEEE; 2020. p. 8110–8119.
   Google Scholar
• Ramesh A, Dhariwal P, Nichol A, et al. Hierarchical text-conditional image generation with clip latents. arXiv Preprint arXiv. 2022;{abs/2204.06125}.
   Google Scholar
• Nichol A, Dhariwal P, Ramesh A, et al. Glide: towards photorealistic image generation and editing with text-guided diffusion models. arXiv Preprint arXiv: 211210741. 2021.
   Google Scholar
• Rombach R, Blattmann A, Lorenz D, et al. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; New Orleans, LA, USA. IEEE; 2022. p. 10684–10695.
   Google Scholar
• Nirkin Y, Masi I, Tuan AT, et al. On face segmentation, face swapping, and face perception. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018); Xi'an, China. IEEE; 2018. p. 98–105.
   Google Scholar
• Korshunova I, Shi W, Dambre J, et al. Fast face-swap using convolutional neural networks. In: Proceedings of the IEEE international conference on computer vision; Venice, Italy. IEEE; 2017. p. 3677–3685.
   Google Scholar
• Natsume R, Yatagawa T, Morishima S. Rsgan: face swapping and editing using face and hair representation in latent spaces. arXiv Preprint arXiv: 1804.03447. 2018;69:1–2.
   Google Scholar
• Natsume R, Yatagawa T, Morishima S. Fsnet: an identity-aware generative model for image-based face swapping. In: Asian Conference on Computer Vision; Perth, Australia. Springer; 2018. p. 117–132.
   Google Scholar
• Masood M, Nawaz M, Malik KM, et al. Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward. Appl Intell. 2022;53(4):3974–4026. doi: 10.1007/s10489-022-03766-z
   Web of Science ®Google Scholar
• Liu MY, Tuzel O. Coupled generative adversarial networks. Adv Neural Inf Process Syst. 2016;29:469–477.
   Google Scholar
• Perarnau G, Van De Weijer J, Raducanu B, et al. Invertible conditional gans for image editing. arXiv Preprint arXiv. 2016;{abs/1611.06355}.
   Google Scholar
• Lample G, Zeghidour N, Usunier N, et al. Fader networks: manipulating images by sliding attributes. Adv Neural Inf Process Syst. 2017;30:5969–5978.
   Google Scholar
• Choi Y, Uh Y, Yoo J, et al. Stargan v2: diverse image synthesis for multiple domains. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Seattle, WA, USA. IEEE; 2020. p. 8188–8197.
   Google Scholar
• Liu M, Ding Y, Xia M, et al. Stgan: a unified selective transfer network for arbitrary image attribute editing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Long Beach, CA, USA. IEEE; 2019. p. 3673–3682.
   Google Scholar
• Suwajanakorn S, Seitz SM, Kemelmacher-Shlizerman I. Synthesizing obama: learning lip sync from audio. ACM Trans Graphics (ToG). 2017;36(4):1–13. doi: 10.1145/3072959.3073640
   Web of Science ®Google Scholar
• Fan B, Wang L, Soong FK, et al. Photo-real talking head with deep bidirectional lstm. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); South Brisbane, QLD, Australia. IEEE; 2015. p. 4884–4888.
   Google Scholar
• Charles J, Magee D, Hogg D Virtual immortality: reanimating characters from tv shows. In: European Conference on Computer Vision; Amsterdam. Springer; 2016. p. 879–886.
   Google Scholar
• Jamaludin A, Chung JS, Zisserman A. You said that?: synthesising talking faces from audio. Int J Comput Vis. 2019;127(11):1767–1779. doi: 10.1007/s11263-019-01150-y
   Web of Science ®Google Scholar
• Vougioukas K, Petridis S, Pantic M End-to-end speech-driven realistic facial animation with temporal gans In: CVPR Workshops; Long Beach, California. IEEE; 2019. p. 37–40.
   Google Scholar
• Garrido P, Valgaerts L, Sarmadi H, et al. Vdub: modifying face video of actors for plausible visual alignment to a dubbed audio track. In Computer graphics forum. Vol. 34. Hoboken, New Jersey, U.S: Wiley Online Library; 2015. pp. 193–204.
   Google Scholar
• KR P, Mukhopadhyay R, Philip J, et al. Towards automatic face-to-face translation. In: Proceedings of the 27th ACM international conference on multimedia. ACM; Nice, France. 2019. p. 1428–1436.
   Google Scholar
• Malik A, Kuribayashi M, Abdullahi SM, et al. Deepfake detection for human face images and videos: a survey. IEEE Access. 2022;10:18757–18775. doi: 10.1109/ACCESS.2022.3151186
   Web of Science ®Google Scholar
• Zhou P, Han X, Morariu VI, et al. Two-stream neural networks for tampered face detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW); Honolulu, HI, USA. IEEE; 2017. p. 1831–1839.
   Google Scholar
• Swapme; 2018. Available from: https://apps.apple.com/us/app/swapme-by-faciometrics.
   Google Scholar
• Face swap; 2016. Available from: https://github.com/MarekKowalski/FaceSwap/.
   Google Scholar
• Khodabakhsh A, Ramachandra R, Raja K, et al. Fake face detection methods: can they be generalized? In: 2018 international conference of the biometrics special interest group (BIOSIG); Darmstadt, Germany. IEEE; 2018. p. 1–6.
   Google Scholar
• Zhou H, Sun Y, Wu W, et al. Pose-controllable talking face generation by implicitly modularized audio-visual representation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Online. IEEE; 2021. p. 4176–4186.
   Google Scholar
• Le TN, Nguyen HH, Yamagishi J, et al. Openforensics: large-scale challenging dataset for multi-face forgery detection and segmentation in-the-wild. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; Montreal, QC, Canada. IEEE; 2021. p. 10117–10127.
   Google Scholar
• Kuznetsova A, Rom H, Alldrin N, et al. The open images dataset v4. Int J Comput Vis. 2020;128(7):1956–1981. doi: 10.1007/s11263-020-01316-z
   Web of Science ®Google Scholar
• Yang X, Li Y, Lyu S Exposing deep fakes using inconsistent head poses. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Brighton, UK. IEEE; 2019. p. 8261–8265.
   Google Scholar
• Fake app; 2019. Available from: https://www.malavida.com/en/soft/fakeapp/.
   Google Scholar
• Korshunov P, Marcel S. Deepfakes: a new threat to face recognition? assessment and detection. arXiv Preprint arXiv. 2018;{abs/1812.08685}.
   Google Scholar
• Rössler A, Cozzolino D, Verdoliva L, et al. Faceforensics: a large-scale video dataset for forgery detection in human faces. arXiv Preprint arXiv: 180309179. 2018.
   Google Scholar
• Rossler A, Cozzolino D, Verdoliva L, et al. Faceforensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF international conference on computer vision; Seoul, Korea (South). IEEE; 2019. p. 1–11.
   Google Scholar
• Thies J, Zollhöfer M, Nießner M. Deferred neural rendering: image synthesis using neural textures. ACM Trans Graph. 2019;38(4):1–12. doi: 10.1145/3306346.3323035
   Web of Science ®Google Scholar
• Dolhansky B, Bitton J, Pflaum B, et al. The deepfake detection challenge (dfdc) dataset. arXiv Preprint arXiv: 200607397. 2020.
   Google Scholar
• Dufour N, Gully A. Contributing data to deepfake detection research. Google AI Blog. 2019;1(3).
   Google Scholar
• Li Y, Yang X, Sun P, et al. Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Seattle, WA, USA. IEEE; 2020. p. 3207–3216.
   Google Scholar
• Jiang L, Li R, Wu W, et al. Deeperforensics-1.0: a large-scale dataset for real-world face forgery detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Seattle, WA, USA. IEEE; 2020. p. 2889–2898.
   Google Scholar
• Huang J, Wang X, Du B, et al. Deepfake mnist+: a deepfake facial animation dataset. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; Montreal, BC, Canada. IEEE; 2021. p. 1973–1982.
   Google Scholar
• Kwon P, You J, Nam G, et al. Kodf: a large-scale korean deepfake detection dataset. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; Montreal, QC, Canada. IEEE; 2021. p. 10744–10753.
   Google Scholar
• Khalid H, Tariq S, Kim M, et al. Fakeavceleb: a novel audio-video multimodal deepfake dataset. arXiv Preprint arXiv: 210805080. 2021.
   Google Scholar
• Li G, Zhao X, Cao Y, et al. Fmfcc-v: an asian large-scale challenging dataset for deepfake detection. In: Proceedings of the 2022 ACM Workshop on Information Hiding and Multimedia Security; Santa Barbara, CA, USA. ACM; 2022. p. 7–18.
   Google Scholar
• Nadimpalli AV, Rattani A. Gbdf: gender balanced deepfake dataset towards fair deepfake detection. arXiv Preprint arXiv: 220710246. 2022.
   Google Scholar
• Faceswap-gan; 2018. Available from: https://github.com/shaoanlu/faceswap-GAN/.
   Google Scholar
• Bacanin N An object-oriented software implementation of a novel cuckoo search algorithm. In: Proc. of the 5th European Conference on European Computing Conference (ECC’11); Wisconsin, United States. ACM; 2011. p. 245–250.
   Google Scholar
• Kingra S, Aggarwal N, Kaur N. Siamnet: exploiting source camera noise discrepancies using siamese network for deepfake detection. Inf Sci. 2023;645:119341. https://www.sciencedirect.com/science/article/pii/S002002552300926X
   Web of Science ®Google Scholar
• Kingra S, Aggarwal N, Kaur N. Siamlbp: exploiting texture discrepancies for deepfake detection. In: Machine Intelligence Techniques for Data Analysis and Signal Processing: Proceedings of the 4th International Conference MISP 2022; Raipur, India, Volume 1; Springer; 2023. p. 443–455.
   Google Scholar
• Xu Y, Raja K, Verdoliva L, et al. Learning pairwise interaction for generalizable deepfake detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision; Waikoloa, Hawaii. IEEE; 2023. p. 672–682.
   Google Scholar
• Dogoulis P, Kordopatis-Zilos G, Kompatsiaris I, et al. Improving synthetically generated image detection in cross-concept settings. In: Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation; New York, United States. ACM; 2023. p. 28–35.
   Google Scholar
• Yu F, Seff A, Zhang Y, et al. Lsun: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv Preprint arXiv: 150603365. 2015.
   Google Scholar
• Wang Z, Bao J, Zhou W, et al. Dire for diffusion-generated image detection. arXiv Preprint arXiv: 230309295. 2023.
   Google Scholar
• Guarnera L, Giudice O, Battiato S. Level up the deepfake detection: a method to effectively discriminate images generated by gan architectures and diffusion models. arXiv Preprint arXiv: 230300608. 2023.
   Google Scholar
• Ni Y, Meng D, Yu C, et al. Core: consistent representation learning for face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops; New Orleans, Louisiana. IEEE; 2022 June. p. 12–21.
   Google Scholar
• Yu CM, Chen KC, Chang CT, et al. Segnet: a network for detecting deepfake facial videos. Multimedia Syst. 2022;28(3):793–814. doi: 10.1007/s00530-021-00876-5
   Web of Science ®Google Scholar
• Kingra S, Aggarwal N, Kaur N. Lbpnet: exploiting texture descriptor for deepfake detection. Forensic Sci Int: Digital Invest. 2022;42:301452. doi: 10.1016/j.fsidi.2022.301452
   Web of Science ®Google Scholar
• Kohli A, Gupta A. Light-weight 3dcnn for deepfakes, faceswap and face2face facial forgery detection. Multimedia Tools Appl. 2022;81(22):31391–31403. doi: 10.1007/s11042-022-12778-3
   Web of Science ®Google Scholar
• Kaddar B, Fezza SA, Hamidouche W, et al. Hcit: deepfake video detection using a hybrid model of cnn features and vision transformer. In: 2021 International Conference on Visual Communications and Image Processing (VCIP); Munich, Germany. IEEE; 2021. p. 1–5.
   Google Scholar
• Kohli A, Gupta A. Detecting deepfake, faceswap and face2face facial forgeries using frequency cnn. Multimedia Tools Appl. 2021;80(12):18461–18478. doi: 10.1007/s11042-020-10420-8
   Web of Science ®Google Scholar
• Volkova S, Bogdanov A. A deep learning approach to face swap detection. Int J Open Inf Technol. 2021;9(10):16–20.
   Google Scholar
• Wodajo D, Atnafu S. Deepfake video detection using convolutional vision transformer. arXiv Preprint arXiv: 210211126. 2021.
   Google Scholar
• Guan W, Wang W, Dong J, et al. Robust face-swap detection based on 3d facial shape information. In: CAAI International Conference on Artificial Intelligence; 2022 Aug 27; Cham: Springer Nature Switzerland 404–415.
   Google Scholar
• Guarnera L, Giudice O, Battiato S Deepfake detection by analyzing convolutional traces. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops; Seattle, WA, USA. IEEE; 2020. p. 666–667.
   Google Scholar
• de Lima O, Franklin S, Basu S, et al. Deepfake detection using spatiotemporal convolutional networks. arXiv Preprint arXiv: 2006 14749. 2020; 1–6.
   Google Scholar
• Montserrat DM, Hao H, Yarlagadda SK, et al. Deepfakes detection with automatic face weighting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops; Seattle, WA, USA. IEEE; 2020. p. 668–669.
   Google Scholar
• Agarwal S, Farid H, El-Gaaly T, et al. Detecting deep-fake videos from appearance and behavior. In: 2020 IEEE international workshop on information forensics and security (WIFS); Online. IEEE; 2020. p. 1–6.
   Google Scholar
• Fernandes S, Raj S, Ortiz E, et al. Predicting heart rate variations of deepfake videos using neural ode. In: Proceedings of the IEEE/CVF international conference on computer vision workshops; Seoul, Korea (South). IEEE; 2019. p. 0–0.
   Google Scholar
• Sabir E, Cheng J, Jaiswal A, et al. Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI). 2019;3(1):80–87.
   Google Scholar
• Nguyen HH, Fang F, Yamagishi J, et al. Multi-task learning for detecting and segmenting manipulated facial images and videos. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS); Tampa, FL, USA. IEEE; 2019. p. 1–8.
   Google Scholar
• McCloskey S, Albright M Detecting gan-generated imagery using saturation cues. In: 2019 IEEE international conference on image processing (ICIP); Taipei, Taiwan. IEEE; 2019. p. 4584–4588.
   Google Scholar
• Nataraj L, Mohammed TM, Manjunath B, et al. Detecting gan generated fake images using co-occurrence matrices. Electron Imaging. 2019;31(5):532–537. doi: 10.2352/ISSN.2470-1173.2019.5.MWSF-532
   Google Scholar
• Yu N, Davis LS, Fritz M Attributing fake images to gans: learning and analyzing gan fingerprints. In: Proceedings of the IEEE/CVF international conference on computer vision; Seoul, Korea (South). IEEE; 2019. p. 7556–7566.
   Google Scholar
• Marra F, Saltori C, Boato G, et al. Incremental learning for the detection and classification of gan-generated images. In: 2019 IEEE international workshop on information forensics and security (WIFS); Delft, Netherlands. IEEE; 2019. p. 1–6.
   Google Scholar
• Kong C, Chen B, Li H, et al. Detect and locate: exposing face manipulation by semantic-and noise-level telltales. IEEE Trans Inf Forensics Secur. 2022;17:1741–1756. doi: 10.1109/TIFS.2022.3169921
   Web of Science ®Google Scholar
• Dang H, Liu F, Stehouwer J, et al. On the detection of digital face manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition; Seattle, WA, USA. IEEE; 2020. p. 5781–5790.
   Google Scholar
• Rathgeb C, Botaljov A, Stockhardt F, et al. Prnu-based detection of facial retouching. IET Biometrics. 2020;9(4):154–164. doi: 10.1049/iet-bmt.2019.0196
   Web of Science ®Google Scholar
• Jain A, Singh R, Vatsa M. On detecting gans and retouching based synthetic alterations. In: 2018 IEEE 9th international conference on biometrics theory, applications and systems (BTAS); California, USA. IEEE; 2018. p. 1–7.
   Google Scholar
• Zhang X, Karaman S, Chang SF Detecting and simulating artifacts in gan fake images. In: 2019 IEEE international workshop on information forensics and security (WIFS); Delft, Netherlands. IEEE; 2019. p. 1–6.
   Google Scholar
• Wang R, Juefei-Xu F, Ma L, et al. Fakespotter: a simple yet robust baseline for spotting ai-synthesized fake faces. arXiv Preprint arXiv: 190906122. 2019.
   Google Scholar
• Tariq S, Lee S, Kim H, et al. Detecting both machine and human created fake face images in the wild. In: Proceedings of the 2nd international workshop on multimedia privacy and security; Toronto, ON, Canada. ACM; 2018. p. 81–87.
   Google Scholar
• Bharati A, Singh R, Vatsa M, et al. Detecting facial retouching using supervised deep learning. IEEE Trans Inf Forensics Secur. 2016;11(9):1903–1913. doi: 10.1109/TIFS.2016.2561898
   Web of Science ®Google Scholar
• Wu HY, Rubinstein M, Shih E, et al. Eulerian video magnification for revealing subtle changes in the world. ACM Trans Graph. 2012;31(4):1–8. doi: 10.1145/2185520.2185561
   Web of Science ®Google Scholar
• Rahman H, Ahmed MU, Begum S, et al. Real time heart rate monitoring from facial rgb color video using webcam. In: The 29th Annual Workshop of the Swedish Artificial Intelligence Society (SAIS), 2–3 June 2016, Malmö, Sweden; Linköping University Electronic Press; 2016. p. 129.
   Google Scholar
• Rezende DJ, Mohamed S, Wierstra D Stochastic backpropagation and approximate inference in deep generative models. In: International conference on machine learning; Beijing, China. PMLR; 2014. p. 1278–1286.
   Google Scholar
• Chen RT, Rubanova Y, Bettencourt J, et al. Neural ordinary differential equations. Adv Neural Inf Process Syst. 2018;31:6572–6583.
   Google Scholar
• Anand A, Labati RD, Genovese A, et al. Age estimation based on face images and pre-trained convolutional neural networks. In: 2017 IEEE symposium series on computational intelligence (SSCI); Honolulu, HI, USA. IEEE; 2017. p. 1–7.
   Google Scholar
• Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett. 2016;23(10):1499–1503. doi: 10.1109/LSP.2016.2603342
   Web of Science ®Google Scholar
• Wiles O, Koepke A, Zisserman A. Self-supervised learning of a facial attribute embedding from video. arXiv Preprint arXiv: 180806882. 2018.
   Google Scholar
• Blanz V, Vetter T. A morphable model for the synthesis of 3d faces. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques; New York, United States. ACM; 1999. p. 187–194.
   Google Scholar
• Giudice O, Guarnera L, Battiato S. Fighting deepfakes by detecting gan dct anomalies. J Imaging. 2021;7(8):128. doi: 10.3390/jimaging7080128
   PubMed Web of Science ®Google Scholar
• Sandler M, Howard A, Zhu M, et al. 2018 ieee/cvf conference on computer vision and pattern recognition; Salt Lake City, UT, USA. 2018;4510–4520.
   Google Scholar
• Deng J, Dong W, Socher R, et al. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition; Miami, FL, USA. IEEE; 2009. p. 248–255.
   Google Scholar
• King DE. Dlib-ml: a machine learning toolkit. J Mach Learn Res. 2009;10:1755–1758.
   Web of Science ®Google Scholar
• Cheng J, Wu J, Leng C, et al. Quantized cnn: a unified approach to accelerate and compress convolutional networks. IEEE Trans Neural Net Learn Syst. 2017;29(10):4730–4743. doi: 10.1109/TNNLS.2017.2774288
   PubMed Web of Science ®Google Scholar
• Isola P, Zhu JY, Zhou T, et al. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Honolulu, HI, USA. IEEE; 2017. p. 1125–1134.
   Google Scholar
• Parkhi OM, Vedaldi A, Zisserman A. Deep face recognition. In: BMVC 2015-Proceedings of the British Machine Vision Conference 2015; Swansea, UK. British Machine Vision Association; 2015.
   Google Scholar
• Amos B, Ludwiczuk B, Satyanarayanan M, et al. Openface: a general-purpose face recognition library with mobile applications. CMU School Comp Sci. 2016;6(2):20.
   Google Scholar
• Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition; Boston, MA, USA. IEEE; 2015. p. 815–823.
   Google Scholar
• Korshunov P, Marcel S Speaker inconsistency detection in tampered video. In: 2018 26th European signal processing conference (EUSIPCO); Rome, Italy. IEEE; 2018. p. 2375–2379.
   Google Scholar
• Boutellaa E, Boulkenafet Z, Komulainen J, et al. Audiovisual synchrony assessment for replay attack detection in talking face biometrics. Multimedia Tools Appl. 2016;75(9):5329–5343. doi: 10.1007/s11042-015-2848-2
   Web of Science ®Google Scholar
• Chintha A, Thai B, Sohrawardi SJ, et al. Recurrent convolutional structures for audio spoof and video deepfake detection. IEEE J Sel Top Signal Process. 2020;14(5):1024–1037. doi: 10.1109/JSTSP.2020.2999185
   Web of Science ®Google Scholar
• Mittal T, Bhattacharya U, Chandra R, et al. Emotions don’t lie: an audio-visual deepfake detection method using affective cues. In: Proceedings of the 28th ACM international conference on multimedia. ACM; Seattle, WA, USA. 2020. p. 2823–2832.
   Google Scholar
• Shahzad SA, Hashmi A, Khan S, et al. Lip sync matters: a novel multimodal forgery detector. In: 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC); Chiang Mai, Thailand. IEEE; 2022. p. 1885–1892.
   Google Scholar
• Wang G, Zhang P, Xie L, et al. An audio-visual attention based multimodal network for fake talking face videos detection. arXiv Preprint arXiv: 220305178. 2022.
   Google Scholar
• Haliassos A, Vougioukas K, Petridis S, et al. Lips don’t lie: a generalisable and robust approach to face forgery detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Nashville, TN, USA. IEEE; 2021. p. 5039–5049.
   Google Scholar
• Agarwal S, Hu L, Ng E, et al. Watch those words: video falsification detection using word-conditioned facial motion. arXiv Preprint arXiv: 211210936. 2021.
   Google Scholar
• Ekman P, Friesen WV. Measuring facial movement. Environ Psychol Nonverbal Behav. 1976;1(1):56–75. doi: 10.1007/BF01115465
   Web of Science ®Google Scholar
• Hannun A, Case C, Casper J, et al. Deep speech: scaling up end-to-end speech recognition. arXiv Preprint arXiv: 14125567. 2014.
   Google Scholar
• Hegde SB, Prajwal K, Mukhopadhyay R, et al. Visual speech enhancement without a real visual stream. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision; Waikoloa, HI, USA. IEEE; 2021. p. 1926–1935.
   Google Scholar