Behavior recognition algorithm based on motion capture and enhancement

Yuqi Yang; Jianping Luo

doi:10.1117/12.2689663

10 August 2023 Behavior recognition algorithm based on motion capture and enhancement

Yuqi Yang, Jianping Luo

Proceedings Volume 12748, 5th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2023); 1274808 (2023) https://doi.org/10.1117/12.2689663
Event: 5th International Conference on Information Science, Electrical and Automation Engineering (ISEAE 2023), 2023, Wuhan, China

Abstract

Motion modeling and temporal modeling are crucial issues for video behavior recognition. When extracting motion information in two-stream network, the optical flow diagram needs to be calculated in advance and the end-to-end training cannot be realized. 3D CNNs can extract spatiotemporal information, but it requires huge computational resources. To solve these problems, we propose a plug-and-play motion capture and enhancement network (MCE) in this paper, which consists of a temporal motion capture module (TMC) and a multi-scale spatiotemporal enhancement module (MSTE). The TMC module calculates the temporal difference of the feature-level and captures the key motion information in the short temporal range. The MSTE module simulates long-range temporal information by equivalent enlarging the temporal sensitive field through multi-scale hierarchical sub-convolution architecture, and then further enhances the significant motion features by referring to the maxpooling branch. Finally, several experiments are carried out on the behavior recognition standard datasets of Something-Something-V1 and Jester, and the recognition accuracy rates are 49.6% and 96.9%, respectively. Experimental results show that the proposed method is effective and efficient.

Citation Download Citation

Yuqi Yang and Jianping Luo "Behavior recognition algorithm based on motion capture and enhancement", Proc. SPIE 12748, 5th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2023), 1274808 (10 August 2023); https://doi.org/10.1117/12.2689663

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available