Real-time action recognition based on enhanced motion vector temporal segment network

Xue Bai; Enqing Chen; Haron Chweya Tinega

doi:10.1117/12.2540268

14 August 2019 Real-time action recognition based on enhanced motion vector temporal segment network

Xue Bai, Enqing Chen, Haron Chweya Tinega

Proceedings Volume 11179, Eleventh International Conference on Digital Image Processing (ICDIP 2019); 111791W (2019) https://doi.org/10.1117/12.2540268
Event: Eleventh International Conference on Digital Image Processing (ICDIP 2019), 2019, Guangzhou, China

Abstract

At present, the method based on two-stream network has achieved good recognition performance in action recognition, however, its real-time performance is obstructed due to the high computational cost of optical flow. Temporal Segment Network (TSN), a successful example based on the two-stream network, achieves high recognition performance but cannot be processed in real time. In this paper, the motion vector TSN (MV-TSN) is proposed by introducing the motion vector into temporal segment networks, which greatly speeds up the processing speed of TSN. In order to solve the problem of performance degradation caused by the motion vectors lacking fine structure information, we propose a knowledge transfer strategy, which initializes the MV-TSN with the fine knowledge learned by optical flow. The experimental results show that the proposed method achieves a comparable recognition performance to the previous state-of-the-art approaches on UCF-101 and HMDB-51, and the processing speed is 206.2 fps, which is 13 times of the original TSN.

Citation Download Citation

Xue Bai, Enqing Chen, and Haron Chweya Tinega "Real-time action recognition based on enhanced motion vector temporal segment network", Proc. SPIE 11179, Eleventh International Conference on Digital Image Processing (ICDIP 2019), 111791W (14 August 2019); https://doi.org/10.1117/12.2540268

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available