CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12083 > Article
Paper
16 February 2022 3DCNN-based mouth shape recognition for patient with intractable neurological diseases
Yuya Nakamura, Takeshi Saitoh, Kazuyuki Itoh
Author Affiliations +
Proceedings Volume 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021); 120832H (2022) https://doi.org/10.1117/12.2623642
Event: Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 2021, Kunming, China
Abstract
Patients with intractable neuropathy may use a mouth-shape character (MSC)-based communication as an alternative to communication when speech, writing, and PC are unavailable. MSC-based communication requires not only a speaker but also a supporter. However, it may not be easy to read the mouth shape due to the skill of the supporter. The communication support system that automatically recognizes the mouth shape of a speaker is expected. This study aims to develop the whole support system, and this paper works on mouth shape recognition. We introduce 3DCNN-based mouth shape recognition. As for the input data of CNN, not only the color image but also the flow image obtained by applying the optical flow is used, and two outputs of two CNN models are integrated. We collected speech scenes of eight patients with intractable neurological diseases and conducted recognition experiments. As a result, an average recognition rate of 77.1% was obtained. Excluding the two patients, one had difficulty recognizing the mouth shape even by a human due to little movement of the mouth, and the other had a problem shooting, and the average recognition rate of 86.6% was obtained. We demonstrated the effectiveness of the proposed method.
© (2022) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Yuya Nakamura, Takeshi Saitoh, and Kazuyuki Itoh "3DCNN-based mouth shape recognition for patient with intractable neurological diseases", Proc. SPIE 12083, Thirteenth International Conference on Graphics and Image Processing (ICGIP 2021), 120832H (16 February 2022); https://doi.org/10.1117/12.2623642
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 8 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Mouth
Telecommunications
Optical flow
Visualization
Facial recognition systems
Speech recognition
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top