Single-shot structured light depth prediction based on deep learning has wide application demands in intelligent manufacturing and defect detection. In the depth prediction area, existing CNN has poor long-range global modeling capabilities, and Transformer architecture requires large computational resources. For the challenge, we first propose and adopt DPM-UNet (Depth Prediction Mamba UNet), an end-to-end structured light depth prediction model from singleshot grating integrating Mamba and U-Net architectures, which fully leverages the long-range capturing capability and low computational performance of the State Space Model, as well as the pixel-by-pixel reconstruction capability of the UNet. Compared with pure CNN and Transformer architecture, DPM-UNet is able to achieve more accurate prediction with a 26.4% error reduction on public datasets. Experiments showed that DPM-UNet is effective in improving the accuracy and robustness of depth maps, and demonstrates remarkable potential in structured light depth prediction tasks.
KEYWORDS: Depth maps, Structured light, Education and training, 3D image processing, Phase reconstruction, Lithium, Fringe analysis, Signal processing, 3D modeling, 3D metrology, Deep learning
In recent years, end-to-end depth map prediction from single-shot fringe modulation images in structured light 3D measurement(F2D) has drawn widespread attention, which has significantly reduced measurement times and eliminated the complex intermediate steps in the traditional method. However, F2D is a long-distance ill-conditioned prediction problem, and it is difficult for existing regression networks to achieve high-precision pixel-by-pixel prediction over long distances in space and time. For the challenge, we propose APS-UNet(Absolute Phase aided Supervision UNet), an endto- end depth maps prediction network supervised by an absolute phase branch. With the core physical process, absolute phase branch as auxiliary supervision, can decompose the one challenging long-distance prediction into two easier shortdistance prediction tasks. Moreover, in the training process, the two branches provide feedback to each other, enhancing the accuracy and robustness of depth prediction. Compared to Res-UNet, APS-UNet demonstrates a 32% decrease in mean absolute error (MAE) based on the real dataset, highlighting the effectiveness of this network.
Fringe projection profilometry (FPP) has been widely applied in industrial 3D measurement due to its high precision and non-contact advantages. However, FPP may encounter fringe saturation in high-reflective scenes, consequently impacting phase computation and introducing measurement errors. To address this problem, an efficient exposure fusion method is proposed in this paper. We propose incorporating complementary gray codes into the multi-exposure fusion method to improve measurement efficiency for high-reflective scenes. After obtaining high-quality wrapped phase by the fused phase-shifting patterns, complementary Gray code patterns are used to assist phase unwrapping to achieve 3D measurement. This method takes advantage of the fast projection speed and edge error elimination capability of complementary Gray code patterns, requiring only one set of patterns to complete phase unwrapping. Compared to the common multi-frequency method, our method reduces the number of projected images and exposure time. Experiments are conducted to demonstrate the feasibility and efficiency of the proposed method.
KEYWORDS: Semantics, 3D mask effects, 3D metrology, Structured light, Shadows, 3D modeling, Education and training, Deep learning, Image processing, Feature extraction
Deep learning-driven structured light 3D measurement has garnered significant attention due to the fast speed, high precision and non-contact characteristic. However, the accurate prediction of edge discontinuity area is still one of the challenges. In single-frame end-to-end absolute phase prediction task, we initially proposed a mask semantic attention network (MSAN) to enhance the edge and whole accuracy. Firstly, mask serves to partition the scene into its background (shadow) and foreground (objects) elements, and it provides semantic attention for the network. Secondly, we designed a mask fusion (MF) module which can effectively integrates feature maps with mask semantics. Based on the MF module and mask semantic information, we developed a U-shaped network architecture, and each layer feature map of the decoder is fused with the input mask adopting the MF module. MSAN improves edge prediction accuracy by explicitly identifying edge regions and drawing the network's attention to the edges and objects rather than shadow areas, enhancing overall prediction accuracy. Validation on real datasets showed that the mean absolute error decreased by 33% and the root mean square error decreased by 76% with MSAN, demonstrating the network's capability to improve both overall and edge precision in structured light deep learning tasks. This advancement significantly benefits the development of high precise and rapid structured light 3D measurement technologies.
KEYWORDS: Phase unwrapping, Deep learning, 3D metrology, Shadows, Semantics, Network architectures, Education and training, Visualization, Time metrology, Phase reconstruction
Single-frame high-precision 3D measurement using deep learning has been widely studied for its minimal measurement time. However, the long physical and semantic distances make the end-to-end absolute phase reconstruction of single-frame grating challenging. To tackle this difficulty, we propose the DSAS-S2AP-X (Dual-Stage Auxiliary Supervision Network for Single-Frame to Absolute Phase Prediction with X) strategy, which includes the secondary highest frequency unwrapped phase and the highest frequency wrapped phase supervision branches. It combines a multi-frequency temporal phase unwrapping model with existing regression networks X (meaning arbitrary). Experimental results have shown that the DSAS-S2AP-ResUNet34 strategy can reduce the mean absolute error (MAE) and root mean square error (RMSE) of the absolute phase by 34.3% and 25.9% respectively based on the ResUNet34.
The three-frequency heterodyne phase shift profilometry is widely used in high-precision 3D reconstruction. However, the high accuracy comes at the cost of requiring many projected frames, which increases measurement time and decreases measurement efficiency. To address this challenge, we propose a rapid, high-precision absolute phase acquisition method called X+1+1, which fully integrates the accuracy advantages of the multi-frequency n-step heterodyne phase-shifting method and the speed advantages of the Modified Fourier transform profilometry (MFTP). The highest frequency gratings use the standard X-step phase-shifting method to determine the wrapped phase, ensuring high unwrapping accuracy and obtaining background light intensity. For intermediate and low frequencies, a single-frame grating and the Backgroundgenerated Modified Fourier transform profilometry (BGMFTP) are used to solve each wrapped phase to reduce the measurement time. Finally, the heterodyne method processes these three-frequency wrapped phases to obtain the absolute phase. Experimental results demonstrated the high accuracy and speed of this method in the 3D measurement process. Compared to traditional Fourier transform profilometry, the X+1+1 method has a 53% improvement in accuracy, while maintains the same level of performance as the three-frequency four-step heterodyne method in continuous non-marginal flat areas and the projection time was reduced by approximately 50%. The proposed X+1+1 method provides a new solution for balancing speed and accuracy in the application and promotion of structured-light 3D measurement.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.