2.1.

Demo System Design

A fast structured light system consisting of a camera (Nikon D50, Tokyo, Japan), a computing unit, and a projector (Sony VPL-DX11, Kōnan Minato, Tokyo, Japan) was implemented to obtain a 3-D model of a lesion in Fig. 1(a). The projector was used as the space modulator in 3-D reconstruction and then as the light source during treatment. The software process is shown in Fig. 1(b), and the flow path is introduced in the following sections.

Fig. 1

(a) System configuration and (b) the software flow chart.

2.2.

Data Acquisition and 3-D Reconstruction

The system should be calibrated to determine the parameters of the camera and the projector [C&P, Fig. 1(b)], which included rotation and translation matrices $(R,T)$ between them, as well as the calibration parameters of the lens in the C&P. A 3-D scanning of the head model was obtained by a series of structured light patterns (e.g., gray code). Images were simultaneously taken from another angle. The coding and decoding of the gray code, which indicate the corresponding relation of the pixels on the image plane, is shown in Fig. 2. Each 3-D coordinate was indexed from its two-dimensional (2-D) coordinates on the camera image plane.²⁶ Hence, the texture of the 2-D image can be mapped onto the 3-D point cloud to form a 3-D model for observation and further operation.

Fig. 2

Coding and decoding of the gray code.

2.3.

Segmentation and Rating of Lesion

Setting the desired irradiance pixel by pixel remains difficult. Topical segmentation and classification in dermatology depend on the morphological changes of skin lesions, which vary with the disease. A superpixel clustering method was proposed for segmentation and rating under different situations.²⁷ First, an original image was transformed to the LAB color space $[l_i,a_i,b_i]$, and parameter $k$ was set to correspond to the number of areas that require treatment. The pixels were then clustered by their distances in the five-dimensional space along with their corresponding coordinates $[x_i,y_i]$ around the initial center pixel of $[x_k,y_k]$, as given by Eq. (1)

2.4.

Light Delivery

Some of the main concerns related to using projectors as illumination sources include irradiation alignment and incident angle calculation; the latter was essential for targeted and homogeneous irradiances in the lesion. A simplified illumination model is demonstrated in Fig. 3. $O_{\mathrm{c}1}{X}_{\mathrm{c}1}{Y}_{\mathrm{c}1}{Z}_{\mathrm{c}1}$ and $O_{\mathrm{p}}{X}_{\mathrm{p}}{Y}_{\mathrm{p}}{Z}_{\mathrm{p}}$ represent the coordinates of the camera and projector, respectively, $P_{\mathrm{c}}$ is the image plane, and $P_{\mathrm{g}}$ is the projector plane. Given the known 3-D coordinates of the skin diseases, the mapping relation from the image plane coordinate $[x_{\mathrm{c}},y_{\mathrm{c}},z_{\mathrm{c}}]$ to the projector coordinates $[x_{\mathrm{p}},y_{\mathrm{p}},z_{\mathrm{p}}]$ can be described as follows:

Fig. 3

A simplified illumination model.

The target irradiance for a practical voxel can be achieved by modifying the gray value of the projection image. For example, the pulse width of modulation in a digital light projector was linearly correlated with the irradiance. The gray value of the corresponding pixel was then calculated upon setting the desired irradiance. An example of a reverse projection image is shown in Fig. 4.

Fig. 4

An example of reverse projection image.

3.1.

3-D Reconstruction and Evaluation

The 3-D reconstruction and evaluation with a resolution of $<1\mathrm{mm}$ in the $x$, $y$, and $z$ axes is illustrated in Fig. 5. The head model was reconstructed by a 3-D scanner (3D CaMega, Boweihengxin Ltd., Beijing, China) for reference. Figure 5(a) shows $\sim 777,117$ and 345,170 vertices that were obtained from our reconstruction in the left (in brown) and right (in black) with abbreviations of 3-D Sca. and 3-D Rec., respectively. The 3-D point clouds were evaluated by MeshLab. Figure 5(b) shows that the Hausdorff distances were $<1\mathrm{mm}$ in most of the vertices and that the gross errors were almost in the boundary with no points in our 3-D reconstruction because of the shadow.

Fig. 5

Three-dimensional (3-D) reconstruction results compared with the data from a 3-D scanner.

3.2.

Segmentation Results

The clustering results for different $k$ and skin diseases are revealed in Figs. 6 and 7, respectively. The original image of Fig. 6 is a phantom of a port-wine stain (PWS), where the red region represents the lesion. The number of segmentations increased with $k$. This result indicates that segmentation is a flexible and convenient clustering method because the threshold or initial conditions need not be set. Zhang^27,28 demonstrated that segmentation can be evaluated by various techniques and that segmentation applications can be considered during selection. Common empirical methods were used to test the segmentation quality in all four experiment groups (Table 1).^29,30 UM is the intraregion uniformity measurement, GC is the gray-level contrast between regions, VC is the interregion vergence contrast, MI(k) is the rate of pixels that should be segmented when they are not with the proposed algorithm, and MII(k) is the rate of pixels that should not be segmented. MI(k) and MII(k) were obtained by comparing the experimental results with the theoretical results by manual specification.

Fig. 6

Original image (a) with clustering results when $k$ is 50 (b) and 200 (c).

Fig. 7

Clustering and segmentation results for some dermatological diseases: (a) herpes zoster, (b) hemangioma cutis, (c) tinea corporis, and (d) leucoderma.

Table 1

Evaluation of the segmentation by different properties.

The severities of skin disease are often hierarchical. For instance, the severity of PWS is graded in six levels. The head model was painted with common colors (e.g., purple, dark violet, light red, and dark red) for demonstration.³¹ The areas with different colors were clustered separately (Fig. 8). The expected light dosimetry can be mapped from the camera plane to the projector plane as described in Sec. 2.4.

Fig. 8

Grading sample for port-wine stain.

3.3.

Illumination Results

Given the 3-D reconstruction of the lesion, we can estimate the surface normal at each patch by its surrounding neighbors and their gravity. The cosines of the incident angles of each patch on the surface with five discrete levels for simplicity are shown in Fig. 9. Only one third of the incident rays are nearly vertical to the surface. Thus, even illumination can hardly be obtained on a curved lesion illuminated directly by a flat source.

Fig. 9

Cosines of incident angles for various lesions.

Internal points can be regarded as the interpolation of boundary points. Thus, the difference between the lesion and the reverse projection can be assessed by the degree of boundary matching. The red region in the left of Fig. 6 represents the targeted lesion. The highlighted areas in Fig. 10 were illuminated to verify the correctness of the reverse projection or treatment illumination. The incidence was well targeted, and the normal areas were protected without any additional work. The borders were matched on the lesion, which verified the accuracy of the segmentation and reconstruction.

Fig. 10

Reverse projection results viewed from different angles.

4.1.

Spectrum of Light Source

Monochromatic light is the optimal choice in PDT, but LED and other nonlaser light sources are also widely used. The depth of light penetration into the tissue is related to the light spectrum. Although projectors are often based on lamps, laser and LED light engines are used in high-brightness projectors. LED or laser projectors can be used with the right color channel, or the light engine can be modified with a correct laser source. Two or three color channels can also be combined by different laser types, which are beneficial in simultaneously treating diseases at different depths.

4.2.

Projector Irradiance

The power density required for PDTs is typically 70 to $100\mathrm{mW}/{\mathrm{cm}}^2$. Light sources with power outputs of 5 to 10 W are suitable for most treatments; several modules are also available. For example, Luminus developed an LED chip PT-120 for a TI DLP® light engine with 5400 lumen at 525 nm. Laser light engines possess a green laser module DSG265 that provides a digital projection of up to 35 W. For the modulator in the projector, the DMD9500 datasheet exhibits a threshold density of up to $20\mathrm{W}/{\mathrm{cm}}^2$ in the visible spectrum. This threshold density corresponds to 48 W for the device, which meets PDT requirements. The proposed digital illumination procedure allowed modulation either by modulating the light source or by modifying the gray value of the pixel.

4.3.

Lesion Identification

The head model painted with different colors verified the correctness of the reverse projection. Although color is a characteristic of lesions, its identification requires professional clinical diagnosis. 3-D reconstruction and segmentation provided a convenient control for treatment, for example, by making the treatment area larger than the lesion-defined area, reserving a specific untreated area, or illuminating different areas at various levels.

4.4.

Optical Parameters of Human Skin

Optical parameters (e.g., specular reflection, absorption coefficient, scattering coefficient, photosensitizer absorption, and the index of refraction) vary from person to person. These parameters are related with thermal damage, pain, and treatment evaluation; hence, optimal protocols in the site are highly difficult. All parameters should be achieved and used to calibrate the gray value of the illumination image for the treatment to provide an accurate and ideal PDT. Protocols of different wavelengths or modes of light modulation can be simultaneously assessed on the same patient to understand the mechanism of PDT.

4.5.

Patient Movement During the Treatment

Patient movement during PDT is too difficult to avoid because of the treatment time and pain. PDT is tiring because it takes $\sim 30\min$ per session. Moreover, the procedure can be very excruciating due to various levels of pain and burning sensations. The proposed procedure allows 3-D reconstruction of the lesion at timed intervals as prescribed by a doctor. Then, the result can be registered and aligned with the original 3-D model. Thus, each treatment dosimetry can be not only reserved but also statistically analyzed and compared.