Transfer learning for micro-expression recognition based on the difference key frame images

Zhihua Xie; Le Wang; Ling Shi; Jiawei Fan; Sijia Cheng

doi:10.1117/12.2611665

11 November 2021 Transfer learning for micro-expression recognition based on the difference key frame images

Zhihua Xie, Le Wang, Ling Shi, Jiawei Fan, Sijia Cheng

Proceedings Volume 12076, 2021 International Conference on Image, Video Processing, and Artificial Intelligence; 120760B (2021) https://doi.org/10.1117/12.2611665
Event: Fourth International Conference on Image, Video Processing, and Artificial Intelligence (IVPAI 2021), 2021, Shanghai, China

Abstract

Micro-expression, revealing the true emotions and motives, attracts extraordinary attention on automatic facial microexpression recognition (MER). The main challenge of MER is large-scale datasets unavailable to support deep learning training. To this end, this paper proposes an end-to-end transfer model for facial MER based on the difference images. Compared with micro-expression dataset, macro-expression dataset has more samples and is easy to train for deep neural network. Thus, we pre-train the resnet-18 network on relatively large expression datasets to get the good initial backbone module. Then, the difference images based on adaptive key frame is applied to get MER related feature representation for the module input. Finally, the preprocessing difference images are feed into the pre-trained resent-18 network for fine-tuning. Consequently, the proposed method achieves the recognition rates of 74.39% and 76.22% on the CASME2 and SMIC databases, respectively. The experimental results show that the difference image between the onset and key frame can improve the transfer training performance on resnet-18, the proposed MER method outperforms the methods based on traditional hand-crafted features and deep neural networks.

Citation Download Citation

Zhihua Xie, Le Wang, Ling Shi, Jiawei Fan, and Sijia Cheng "Transfer learning for micro-expression recognition based on the difference key frame images", Proc. SPIE 12076, 2021 International Conference on Image, Video Processing, and Artificial Intelligence, 120760B (11 November 2021); https://doi.org/10.1117/12.2611665

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Databases

Image fusion

Data modeling

Video

Performance modeling

Data fusion

Feature extraction

Show All Keywords

Keywords/Phrases

Search In:

Publication Years