Action-guided CycleGAN for Bi-directional Video Prediction

Abstract

Most of the earlier proposed action prediction methods, such as PoseGAN, C2GAN, etc. are limited to unidirectional prediction and not intended to perform the desired class of action. In this paper, we propose an action-guided Cycle Generative Adversarial Network (AGCGAN), which is a bi-directional video prediction model to anticipate future frames from current visual frames and vice versa. The proposed CycleGAN architecture consists of two generators and two discriminators along with two additional keypoint detectors to make keypoint loss measurements. The adversarial and cycle losses perform the appearance modelling whereas the keypoint loss provides the necessary motion correction. We evaluated the effectiveness of our proposed method on the UT-Interaction dataset. Furthermore, we also tested our model on the standard Market-1501 dataset against the state-of-the-art schemes. Results show that our proposed method provides comparable visual quality with the state-of-the-art methods.

KEYWORDS:

• Action prediction
• generative network
• keypoint loss
• video prediction

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Amit Verma

Amit Verma is currently a PhD research scholar in the Department of Electronics & Communication at National Institute of Technology Raipur, India. His research interests include image processing, computer vision, and neural networks. Corresponding author. Email: [email protected]

Toshanlal Meenpal

Toshanlal Meenpal is currently an associate professor in the Department of Electronics & Communication at the National Institute of Technology Raipur, India. He obtained his PhD from Bhabha Atomic Research Centre (BARC), Mumbai under the aegis of HBNI University, Mumbai. He did his master's degree in automation and computer vision engineering from the Indian Institute of Technology, Kharagpur in the year 005. Before switching to academics he has also worked for 5 years as design engineer at ST Microelectronics and Nvidia Graphics in the Multimedia playback-related R&D groups. His research interests include multimedia security techniques like digital watermarking, steganography, and cryptography as well as image processing and image analysis. Email: [email protected]

Bibhudendra Acharya

Bibhudendra Acharya was born in India, on June 30, 1978. Graduated from Dr. B. A. Marathawada University, Aurangabad, India in electronics and telecommunication engineering in the year 2002 and did his MTech from the National Institute of Technology, Rourkela, Odisha, India in telematics and signal processing in the year 2004 and the PhD degree from the National Institute of Technology, Rourkela, Odisha, India, in 2015. He is currently serving as an associate professor in the Department of Electronics & Communication, NIT, Raipur. He has more than 40 research publications in national/international journals and conferences. His research areas of interest are cryptography and network security, microcontroller and embedded systems, signal processing, and soft computing. Email: [email protected]