The performance of facial action unit (AU) detection is often limited owing to the lack of annotated AU data and data imbalance. The data scarcity problem can be partially mitigated by abundant expression data, as AU detection and facial expression recognition (FER) are closely related. Accordingly, in this study, FER and AU detection are trained jointly in a multi-task learning framework, in which FER serves as an auxiliary task for AU detection by providing supplementary information. Meanwhile, we propose to use an attention gate unit between the two tasks to flexibly select valuable information from each other. To address the model bias issue caused by AU data imbalance, a smooth class-weighted loss is adopted to alleviate the dominance of negative AU classes. The best average F1-score obtained using our approach on the BP4D dataset is 63.5%, which is very close to the state-of-the-art performance and exceeds the single task baseline by 3.2%.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.