SpecFlow: an end-to-end framework for specular reflection restoration in endoscopy videos using flow-guided video completion

Haoli Yin; Rachel L. Eimen; Daniel Moyer; Audrey K. Bowden

doi:10.1117/12.2650509

6 March 2023 SpecFlow: an end-to-end framework for specular reflection restoration in endoscopy videos using flow-guided video completion

Haoli Yin, Rachel L. Eimen, Daniel Moyer, Audrey K. Bowden

Author Affiliations +

Proceedings Volume PC12368, Advanced Biomedical and Clinical Diagnostic and Surgical Guidance Systems XXI; PC123680G (2023) https://doi.org/10.1117/12.2650509
Event: SPIE BiOS, 2023, San Francisco, California, United States

Abstract

Specular reflections (SR) commonly found in endoscopy videos can severely disrupt a surgeon’s observation and judgment, but existing methods to inpaint SR regions can result in false clinical interpretations. Therefore, we propose an end-to-end pipeline termed SpecFlow to detect and restore SR regions from endoscopy videos. Our proposed SpecFlow consists of two phases: detection using a reduced U-net model and a novel restoration method using optical flow-guided color propagation. Our detection pipeline achieves a competitive 82.8% Dice score with only 14ms of computational time (near real-time), and our restoration pipeline successfully incorporates temporal information for more accurate restorations.

Conference Presentation

Citation Download Citation

Haoli Yin, Rachel L. Eimen, Daniel Moyer, and Audrey K. Bowden "SpecFlow: an end-to-end framework for specular reflection restoration in endoscopy videos using flow-guided video completion", Proc. SPIE PC12368, Advanced Biomedical and Clinical Diagnostic and Surgical Guidance Systems XXI, PC123680G (6 March 2023); https://doi.org/10.1117/12.2650509

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
PRESENTATION

WATCH
PRESENTATION SAVE TO MY LIBRARY

GET CITATION

KEYWORDS

Video

Endoscopy

Specular reflections

Optical flow

Visual process modeling

Image segmentation

Surgery

Show All Keywords

Keywords/Phrases

Search In:

Publication Years