1.

Introduction

Biological vision in nature provides compelling opportunities for new functionality and small form factors in diverse imaging systems.^1–7 In particular, about one million species of insects possess remarkable features such as wide viewing angle, large depth of field (DOF), low optical aberration, and polarization sensitivity.^8,9 Compound eyes often exhibit a small number of retinula cells in a single channel, called ommatidium. In contrast, Xenos peckii has an unusual design of photoreceptors on an individual eyelet.^10,11 Most compound eye cameras emulate their anatomical configuration of hemispherically arranged lenses and photoreceptors for large field-of-view (FOV) imaging.^12–14 However, the lens systems have technical limitations in low image resolution and complex fabrication processes. Artificial compound eye cameras inspired by Xenos peckii eye mainly used commercialized planar image sensors to comprise delicately defined multiple pixels in a single channel and achieve relatively high image resolution,^7,15–19 whereas the FOV of these type-cameras still remains narrow.

Inconvenient tools such as mirrors and cheek retractors are usually used for conventional dental examinations. The traditional dental tools have limitations that make it difficult for patients to directly check the condition of their teeth and require a wide opening of the mouth that causes discomfort to the patient. Compact intraoral dental cameras have been developed to overcome the discomfort and collect dental conditions compared with historical data.^20,21 The intraoral cameras also offer several functions such as teeth surface reconstruction,^22–24 and dental health inspection.²⁵ The dental camera accurately captures the structures and diseases of teeth through the compact camera integration and image reconstruction.^26,27 For instance, the intraoral scanners often provide the anterior and posterior teeth arrangement as well as the position of the dental cavity.^28,29 Dentists require teeth images from various angles such as right/left buccal and maxillary/mandibular occlusal for accurate diagnosis of teeth.^30,31 However, previously developed intraoral cameras have technical bottlenecks such as large minimum object distance, thick total track length, narrow viewing angle, and the absence of functionality fusion.^32,33

Here, we report an ultrathin intraoral camera for assorted functional dental imaging, inspired by the vision scheme of Xenos peckii’s eye, as shown in Fig. 1(a). The biologically inspired intraoral camera (BIOC) is fully packaged with a convex-concave lens, LED, and inverted microlens arrays (iMLAs) on a single complementary metal-oxide-semiconductor (CMOS) image sensor. The convex-concave lens collects light from wide-angle directions, coupled to the iMLAs, thus increases the viewing angle of BIOC about 1.7 times. A short focal length of iMLAs allows a wide DOF of BIOC for all-in-focus imaging, ranging from the surface of teeth to the palate. The fabrication steps of BIOC involve the microfabrication of iMLAs with optical block layers and the camera assembly, as shown in Fig. 1(b). Figure 1(c) shows the ultrathin array camera (UAC) packaged on a flexible printed circuit board (FPCB). The UAC, LED, and circuit components were mounted inside an anodized aluminum handpiece holder with 3-mm thickness to prevent deformation by sterilization. As shown in Figs. 1(d) and 1(e), the fully-assembled BIOC has a physical dimension of 7.8 mm in thickness and 25 mm in diameter and thus facilitates the intraoral operation for multifunctional dental imaging assembly. The thickness of BIOC is about 2.3 times as thin as that of a conventional wide-angle single-lens camera (WSLC, Raspberry Pi camera with 140-deg wide-angle lens, focal length: 2.72 mm).

Fig. 1

BIOC. (a) Schematic illustrations of BIOC, comprising a convex-concave lens, cover frames, LED, iMLAs, spacers, and CMOS image sensor on a FPCB. (b) Microfabrication steps and camera assembly of BIOC, using metal evaporation, lift-off, repetitive photolithography, and upside-down reflow. The cover frames were assembled through an epoxy adhesive for waterproofing. (c) The captured image of an UAC sensor packaged with the iMLAs and an image sensor mounted on the FPCB. (d) The comparison image of thickness between the BIOC and WSLC in the side-view. (e) Top view images of fully-assembled BIOC.

2.

Materials and Methods

The optical block layers consist of a 100-nm thick chromium (Cr) layer and double black SU-8 layers. LOR resist and AZ GXR-601 were patterned on a 4-inch borosilicate glass wafer, and the Cr film was deposited through evaporation on the patterns. The remaining resist patterns were removed through a lift-off process. A $5\text{-}\mu \mathrm{m}$ thick black SU-8 (Gersteltec, GMC 1040) and a $25\text{-}\mu \mathrm{m}$ thick SU-8 2025 were defined by repeating photolithography on the Cr patterns. AZ9260 resist was patterned into a cylindrical shape on the block layers, and an upside-down reflow process melted the cylinder patterns to form microlens shapes on a 180-deg convection oven. The fabricated wafer with the iMLAs was diced to $8\mathrm{mm}\times 8\mathrm{mm}$, and the lens plate was packaged with an image sensor mounted on a flexible board (OmniVision OV5647, 5 M pixels, pixel size: $1.4\mu \mathrm{m}\times 1.4\mu \mathrm{m}$) by using stacked alumina spacers with an epoxy adhesive. For the camera assembly, the convex-concave lens and LED were bonded on the cover frame by using UV curable adhesive (Norland, NOA63). The cover frames were combined using high shear and peel strength epoxy adhesive (3M, DP420) to prevent disinfectant penetration.

Multiple channel images captured by the BIOC were integrated through a high dynamic range imaging technique.³⁴ The array images were cropped to each channel image, and the cropped images were aligned by matching the outer line of the images. The registered images were integrated by calculating a weighted average of each pixel. The weight factor was determined through the saturation and contrast of images.

3.

Results and Discussion

The DOF of BIOC was also characterized by a 3D dental phantom, as shown in Fig. 2(a). The experimental results show that the WSLC captures clear images at 3 cm from the central incisor of dental phantom but defocused images at close- or far distances over the DOF, for instance, 1 or 5 cm, respectively, as demonstrated in Fig. 2(b). The reason for image defocusing is because the image plane distance of WSLC rapidly transits according to the object distance, and the DOF of WSLC becomes shallow. Conventional intraoral cameras are often integrated with lens actuators to adjust a focusing plane, however, the actuator increases the total camera thickness and the actuator mounted camera still captures blurred images at a short distance less than about 1 cm.³⁵ In contrast, the BIOC of a short focal length exceptionally exhibits a wide DOF, resulting in clear focused images at all object distances, as shown in Fig. 2(c). Figure 2(d) indicates that the BIOC clearly demonstrates a constant MTF50 along with the object distances, whereas the WSLC shows a significant decrease in MTF50 at object distances outside the DOF. As shown in Fig. 2(e), the array images of each channel acquire nearly similar scenes due to the wide viewing angle, thus multiple channel images were combined into a single image. The color difference of array images obtained from each channel is caused by the difference between the chief ray angles of the image sensor and that of iMLAs. Image reconstruction by merging multiple images slightly compensates for the color differences. In addition, a convex-concave lens on UAC substantially increases the FOV, as shown in Figs. 2(f) and 2(g). The BIOC captures the entire teeth of the 3D dental phantom due to a large FOV of 140-deg, whereas the UAC has the 80-deg in FOV, observing the partial teeth. The result indicates that the BIOC has a similar FOV to the WSLC with 140 deg. Note that the thickness of BIOC is 2.3 times smaller than that of WSLC.

Fig. 2

Dental phantom imaging through the WSLC and the BIOC. (a) Demonstration of phantom dental imaging through the fully-assembled BIOC. Optical images of the central incisor and the second molar were captured by (b) the WSLC and (c) the BIOC, depending on the object distances. The WSLC captures blurred images at close (1 cm) and far (5 cm) distances due to a limited depth-of-field. The BIOC provides clear images at all the different distances. (d) Comparison of MTF 50 between the captured images from the WSLC and the BIOC, which shows that dental images captured by the BIOC have a constant MTF value over the object distance. (e) Array image of dental phantom captured by the BIOC, and a reconstructed image through integral imaging. Optical images of the bottom dental arch were captured by (f) the UAC and (g) BIOC. The BIOC clearly observes the whole arrangement of the bottom dental arch compared to the UAC.

The BIOC delivers assorted functional images of human teeth after reconstructing captured array images. The human teeth imaging was performed after the KAIST institutional review board (IRB) approval. Three different cameras, including types A, B, and C, were compared for clinical trials, as shown in Fig. 3. The usability testing result shows that the A- and B-type intraoral cameras are convenient for observing the occlusal surface of teeth but inconvenient for investigating the inside of central incisors. The C-type camera is relatively suitable for observing all parts of teeth. For instance, it can capture the occlusal surface of second molars, anatomically located far away from incisors.

Fig. 3

Various types of intraoral camera prototypes for usability testing. Types A and B are slim intraoral cameras without the convex-concave lens and with the convex-concave lens, respectively. Type C is an intraoral camera with a dental mirror shape. The thickness of type A camera without the convex-concave lens is 5.5 mm, and that of types B and C is 7.8 mm.

The BIOC captures various human teeth images, such as the second molars, the second premolars, and the central incisors, as shown in Figs. 4(a)–4(d). The second premolars are also captured from the buccal surface through a narrow gap between the cheek and the teeth. The integral imaging was conducted to improve color and hue contrast by combining array images into a single image. Figure 4(e) shows that the dental plaque image noticeably distinguishes the boundary of tooth and plaque after the image reconstruction of captured array images of BIOC. Acquired images were compared by expressing the color difference between a tooth, a gum, and plaque in a 1931 chromaticity diagram. The reconstructed image significantly enlarges the area of the color gamut by $>2.23$ times over a single-channel image, as demonstrated in Fig. 4(f). The experimental result also indicates that the color gamut area of the reconstructed image is comparable to that of WSLC, as shown in Fig. 4(g). The results indicate that the reconstructed image of BIOC has a color perception ability similar to that of a conventional single-lens camera image.

Fig. 4

Clinical trial for dental imaging through the BIOC. (a) Schematic illustration of dental arrangement for displaying the observation locations. Human dental images of (b) the second molar, (c) the second premolar, and (d) the central incisor were captured by the BIOC. (e) Human central incisor images are captured by (i) the single channel of BIOC (BIOC_S), (ii) reconstructed image from array images (BIOC_R), and (iii) the image of WSLC. (f) The CIE-1931 chromaticity diagram comparing color coordinates after the image reconstruction. The area of the color diagram is substantially increased by 2.23 times after the image reconstruction. (g) The color chromaticity diagram comparing images captured by the WSLC and the reconstructed image.

The visual disparity is observed between the anterior teeth’s top and bottom channel images in Fig. 5(a). The red-cyan anaglyph image is acquired by overlapping the top and bottom channel images as demonstrated in Fig. 5(b). The corresponding 3D color depth map exhibits the precise depth resolution of $84\mu \mathrm{m}$, and the anterior teeth have a depth difference of 2 mm, as shown in Fig. 5(c). This depth disparity can be clinically used to determine malocclusion through the depth difference between the teeth of two arches.³⁶ The autofluorescence imaging of BIOC further visualizes the plaque boundary under UV illumination. Oral bacteria or plaque can be observed with UV light due to the autofluorescence of porphyrin, i.e., organic compounds contained in dental biofilm, as shown in Fig. 5(d). The porphyrin emits red fluorescence when a UV light source is irradiated. The fluorescence image of dental plaque is captured through the BIOC, as demonstrated in Fig. 5(e). The fluorescence imaging of BIOC can be used to analyze the plaque area without errors clearly as well as widely observe the overall teeth. In addition, the dental inspection requires patient cooperation, such as widely opening the mouth. In particular, the maxillary and mandibular second molars, located distally from the incisors, have difficulty capturing clear images due to a lack of anatomical space.^37,38 However, the BIOC overcomes the anatomical limitations through its thinness and allows capturing clear pictures with minimal discomfort.

Fig. 5

Multifunctional dental imaging. (a) Anterior dental images were extracted from the top and bottom channels of BIOC. (b) A red-cyan anaglyph image was acquired by merging the top and bottom channel images. (c) Reconstructed 3D depth color map of anterior teeth, showing the depth resolution of $84\mu \mathrm{m}$ as well as the depth difference of 2 mm between the top and bottom dental arches. (d) The principle of fluorescence dental imaging. The red fluorescence is emitted by autofluorescence of porphyrin when the teeth with porphyrin compound are irradiated with a UV light source. (e) A captured fluorescence dental image was recorded by the BIOC with a UV light source.

4.

Conclusions

We have successfully demonstrated a multifunctional intraoral camera inspired by the vision scheme of the Xenos peckii eye. The BIOC features the convex-concave lens, the cover frames, the LED, and the ultrathin camera. The BIOC allows all-in-focus dental imaging and multifunctional imaging by reconstructing captured array images. The experimental results exhibit that reconstructed images clearly increase the area of color space by 2.23 times and offer a 3D depth map with $84\text{-}\mu \mathrm{m}$ depth accuracy. Also, the UV fluorescence dental imaging of BIOC clearly visualizes dental plaques. This biomedical camera module provides opportunities for exploring multifunctional imaging of diverse anatomical structures or epidermal surfaces through narrow natural orifices of humans.