2.1.

Forward Model of a Diffractive Visual Processor under Spatially Coherent Illumination

The propagation of a coherent field from the $l^{\text{th}}$ to the ${(l+1)}^{\text{th}}$ diffractive plane is calculated using the angular spectrum method:²³

Here, $u_i^0$ is the complex field at the input field of view (FOV):

2.2.

Forward Model of a Diffractive Visual Processor under Spatially Partially Coherent Illumination

To model the propagation of a partially coherent field, we define the phase profile ${\phi }_i(x,y)$ of the incident field as^24–26

Using Eq. (7), for a given combination of $\mu$, ${\sigma }_0$, and $\sigma$ values, we numerically approximated $C_{\varphi }$ based on 2048 randomly selected phase profiles, ${\phi }_i(x,y)$. In this study, we used $C_{\varphi }$ values ranging from $\sim 0.5\lambda$ to $\sim 3.0\lambda$ with increments of $\sim 0.5\lambda$, corresponding to $\sigma$ values of $\sim 1.9\lambda$, $\sim 2.4\lambda$, $\sim 3.9\lambda$, $\sim 4.5\lambda$, and $\sim 5.0\lambda$, respectively. Figure S1 in the Supplementary Material provides more information on the generation of these phase profiles with the above-described parameters.

The time-averaged intensity $\hat{O}(x,y)$ of the diffractive visual processor under spatially partially coherent illumination is calculated as

Due to limited computing resources, we used $N_{\varphi }^{\text{test}}=2048$ in the blind testing stage of each trained diffractive model.

2.3.

Training Loss Function

The partially coherent unidirectional diffractive processors presented in this work aim to align the forward output intensity ${\widehat{O}}_{\text{for}}$ closely with the target intensity profile, $I$, while suppressing the backward output intensity, ${\widehat{O}}_{\text{back}}$. To optimize the design of a partially coherent unidirectional diffractive imager, the loss function is composed of a forward loss function ${\mathcal{L}}_F({\widehat{O}}_{\text{for}},I)$ and a backward loss function ${\mathcal{L}}_B({\widehat{O}}_{\text{back}},I)$, i.e.,

Here, the normalized mean square error (NMSE) is used to penalize the structural differences between ${\widehat{O}}_{\text{for}}$ and target intensity $I$, defined as

The Pearson correlation coefficient (PCC) measures the linear correlation between the input intensity, $I$, and the forward output intensity, ${\widehat{O}}_{\text{for}}$, resulting in a value between −1 and 1; it is calculated by

The third term in Eq. (10) refers to a diffraction efficiency-related loss function, which is used to increase the forward diffraction efficiency, $\eta ({\widehat{O}}_{\mathrm{for}},I)$, which is calculated by

The backward loss function in Eq. (9) contains two parts:

2.4.

Performance Evaluation Metrics

To evaluate the performance of each unidirectional diffractive imager, four different metrics are used:

When calculating the backward metrics, such as $\mathrm{PCC}({\widehat{O}}_{\text{back}},I)$, $\eta ({\widehat{O}}_{\text{back}},I)$, and $\mathrm{PSNR}({\widehat{O}}_{\text{back}},I)$, we simply replaced ${\widehat{O}}_{\text{for}}$ with ${\widehat{O}}_{\text{back}}$ while keeping the other parameters unchanged.

3.1.

Unidirectional Imager Design under Spatially Partially Coherent Illumination

Figure 1(a) illustrates the concept of a unidirectional imager under a spatially partially coherent, monochromatic illumination at a wavelength of $\lambda$. The processor is designed to implement a unidirectional imaging task: high structural similarity and high diffraction efficiency for the forward operation A → B [blue line in Fig. 1(a)] together with a distorted image and reduced diffraction efficiency at the backward operation B → A [brown line in Fig. 1(a)]. We used four diffractive layers, axially spaced by $d$ in this design. Each diffractive layer contains $200\times 200$ diffractive features used to modulate the phase of the transmitted optical field, with each feature having a lateral size of $0.53\lambda$ and a trainable phase value covering $[0,2\pi ]$. The transmission functions of these diffractive features are optimized using a training dataset composed of MNIST handwritten digits (see Sec. 2). An engineered loss function guides the optimization of the unidirectional visual processor toward achieving two primary goals: (1) for A → B, it aims to minimize the structural differences between the output images and the ground-truth images using the NMSE and PCC, while concurrently maximizing the forward diffraction efficiency; (2) for B → A, the loss function maximizes the differences between the backward output images and the ground-truth images and simultaneously minimizes the backward diffraction efficiency (refer to Sec. 2 for details).

Fig. 1

Concept of a unidirectional diffractive imager with partially coherent illumination. (a) Schematic of unidirectional imager under partially coherent, monochromatic illumination with a wavelength of $\lambda$. The unidirectional diffractive processor reproduces the input object’s image in the forward propagation direction (blue line from FOV A to FOV B), while suppressing the image formation in the backward propagation direction (brown line from FOV B to FOV A). This design includes four phase-only diffractive layers axially spaced by $d$, each containing $200\times 200$ diffractive features that modulate the phase of the transmitted optical field. (b) Six sets of random phase profiles of partially coherent illumination, each containing $N_{\varphi }^{\text{test}}$ phase profiles. Each set corresponds to one specific correlation length, $C_{\varphi }^{\text{test}}$, varying from $\sim 0.5\lambda$ to $\sim 3\lambda$.

For spatially partially coherent monochromatic illumination, we used the phase correlation length, $C_{\varphi }$, to quantify the degree of the spatial coherence of the source.^24,29 Figure 1(b) illustrates some examples of the random phase profiles of partially coherent illumination with different correlation lengths, varying from $\sim 0.5\lambda$ to $\sim 3\lambda$; also see Sec. 2 and Fig. S1 in the Supplementary Material for details. During the training of a unidirectional diffractive imager, for each input object, we use $N_{\varphi }^{\text{train}}$ different random phase profiles that follow a given correlation length $C_{\varphi }$ at the input plane; the time-averaged intensity of the resulting complex output fields for these $N_{\varphi }^{\text{train}}$ different phase profiles is then used to optimize the unidirectional imager performance based on our training loss function. Details about the optical model of a diffractive unidirectional imager, the training strategy, and loss functions can be found in Sec. 2.

Figure 2(a) illustrates the layout of the partially coherent unidirectional imager design with a compact axial length of $\sim 75\lambda$, which was optimized using $N_{\varphi }^{\text{train}}=16$ and $C_{\varphi }^{\text{train}}=2.5\lambda$. This deep-learning-optimized diffractive design is composed of four spatially engineered diffractive layers, which are displayed in Fig. 2(b). We blindly tested this partially coherent unidirectional imager using 10,000 handwritten digits never seen during the training process; for each unknown input test object, 2048 random phase profiles, each with a phase correlation length of $2.5\lambda$, were used to obtain the time-averaged intensity at the output plane (i.e., $N_{\varphi }^{\text{test}}=2048$—which remained the same throughout our paper). We refer to this testing scheme as “internal generalization” from the perspective of the spatial coherence properties of the illumination light, since we maintained the same statistical phase correlation length in the testing stage as used in the training, i.e., $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$. Some of these blind testing results are illustrated in Fig. 2(c). The first row in Fig. 2(c) shows the input amplitude objects used for both the forward and backward directions. The following two rows in Fig. 2(c) show the forward output images and the backward output images. To better illustrate the details of the backward output images, the last row in Fig. 2(c) further displays higher contrast images of the backward direction B → A, covering a lower intensity range. These visual comparisons clearly demonstrate the success of the partially coherent diffractive unidirectional imager design, reproducing the input images in the forward direction while blocking them in the backward direction—as desired. We also quantified the performance of this unidirectional visual processor using various metrics, including PCC, diffraction efficiency, and PSNR of the forward and backward directions, as shown in Figs. 2(d)–2(f). These metrics are calculated across 10,000 MNIST test objects, never used in training. The PCC values for the forward and backward are $0.9541\pm 0.0239$ and $0.1464\pm 0.1005$, respectively. Furthermore, the forward diffraction efficiency, $85.59\pm 0.05\%$, is about fourfold higher than the backward diffraction efficiency, $22.41\pm 0.03\%$, as shown in Fig. 2(e). A similar desired performance is also observed between the forward PSNR and backward PSNR ($18.46\pm 1.84$ and $8.79\pm 2.01$, respectively), as shown in Fig. 2(f).

Fig. 2

Performance of a partially coherent unidirectional imager. (a) Physical layout of a four-layer unidirectional imager design. (b) Optimized phase profiles of the diffractive layers in a unidirectional imager trained with $N_{\varphi }^{\text{train}}=16$ and $C_{\varphi }^{\text{train}}=2.5\lambda$. (c) Blind testing results of the unidirectional imager with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$ and $N_{\varphi }^{\text{test}}=2048$. The first three rows display the input amplitude objects, forward output images, and backward output images, all using the same intensity range. For comparison, the last row displays the backward output images with increased contrast. Note that both the forward propagation and backward propagation use the same input amplitude objects as displayed in the first row. (d)–(f) Performance evaluation of the diffractive unidirectional imager using 10,000 MNIST test images with PCC, diffraction efficiency, and PSNR metrics.

3.2.

Impact of $N_{\varphi }^{\text{train}}$ on the Performance of Partially Coherent Unidirectional Diffractive Imagers

To explore the influence of $N_{\varphi }^{\text{train}}$ on the performance of partially coherent unidirectional imagers, we trained seven diffractive processors with different $N_{\varphi }^{\text{train}}$ values ranging from 1 to 64 (see Fig. 3). All these diffractive models were trained and tested using the same partially coherent illumination with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$, and in our blind testing, we used $N_{\varphi }^{\text{test}}=2048$. We observed that the diffractive processors trained with a larger $N_{\varphi }^{\text{train}}$ exhibited improved asymmetric imaging performance between the forward and backward directions, as quantified in Figs. 3(a)–3(c). To better quantify the unidirectional imaging capability of a partially coherent diffractive processor, we defined an FOM by calculating the sum of the diffraction efficiency ratio and the image PSNR ratio between the forward and backward imaging directions (see Sec. 2). The resulting FOM is reported as a function of $N_{\varphi }^{\text{train}}$ in Fig. 3(d), which reveals that increasing $N_{\varphi }^{\text{train}}$ improves the FOM of the unidirectional imager up to $N_{\varphi }^{\text{train}}\sim 16$, and beyond 16, the performance differences among different designs diminish. Figure 3(e) further presents the blind testing results for different $N_{\varphi }^{\text{train}}$ values. The first three rows of Fig. 3(e) display the input amplitude objects (used for both the forward and backward directions), the forward outputs, and the backward outputs, respectively. The last row in Fig. 3(e) also displays the backward outputs with increased contrast. As desired, all the backward output images of these diffractive models appear as noise with poor diffraction efficiency. Based on these performance analyses, we conclude that $N_{\varphi }^{\text{train}}=16$ is sufficient to design/train a partially coherent diffractive unidirectional imager, and therefore, we adopted $N_{\varphi }^{\text{train}}=16$ in subsequent diffractive models to speed up the training process.

Fig. 3

Influence of $N_{\varphi }^{\text{train}}$ on the performance of partially coherent unidirectional imagers. (a)–(d) Performance analysis of partially coherent unidirectional diffractive imagers with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$, $N_{\varphi }^{\text{test}}=2048$, and different $N_{\varphi }^{\text{train}}$ values ranging from 1 to 64. The performance in each case was evaluated using 10,000 MNIST test images with PCC, diffraction efficiency, PSNR, and FOM metrics. (e) Examples of the blind testing results with different $N_{\varphi }^{\text{train}}$ values. The first three rows display the input amplitude objects, forward output images, and backward output images, all using the same intensity range. For comparison, the last row shows the backward output images with increased contrast.

Furthermore, Fig. S2 in the Supplementary Material illustrates the numerical performance comparison of a diffractive model trained with $N_{\varphi }^{\text{train}}=16$ and tested under different $N_{\varphi }^{\text{test}}$ values, revealing similar performance results for $N_{\varphi }^{\text{test}}$ ranging from 16 to 2048. In the rest of our blind testing analysis, we used $N_{\varphi }^{\text{test}}=2048$, since the testing time is negligible compared to the training process.

3.3.

Impact of $C_{\varphi }^{\text{train}}$ on the Performance of Partially Coherent Unidirectional Diffractive Imagers

In previous subsections, we analyzed the performance of partially coherent unidirectional diffractive imager designs with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$. To understand the impact of the $C_{\varphi }^{\text{train}}$ on the performance of unidirectional imaging, we trained six additional diffractive visual processors under partially coherent illumination with different correlation lengths ranging from $\sim 0.5\lambda$ to $\sim 3\lambda$. Each diffractive visual processor was tested with the same phase correlation length as used in the training, i.e., $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}$ (see Fig. 4). To evaluate the performances of these diffractive visual processors in both the forward and backward directions, we calculated various metrics, such as PCC, diffraction efficiency, PSNR, and FOM, using 10,000 MNIST test images [see Figs. 4(a)–4(d)]. Figure 4(b) shows a sharp decline in the diffraction efficiency when illuminated by partially coherent light with shorter phase correlation lengths, suggesting that, in these cases, the optimization was empirically dominated by the structural loss term rather than the diffraction efficiency-related loss term. A visualization of the blind testing results is also displayed in Fig. 4(e), where the first three rows depict the input amplitude objects, the forward outputs, and the backward outputs, respectively. For better comparison, the last row in Fig. 4(e) also shows the backward output images with increased contrast. Note that we use a reduced intensity range for diffractive models with smaller phase correlation lengths due to their poor diffraction efficiency, as indicated by the red box in Fig. 4(e).

Fig. 4

Influence of $C_{\varphi }^{\text{train}}$ on the performance of partially coherent unidirectional imagers. (a)–(d) Performance analysis of partially coherent unidirectional diffractive imagers with different ${\mathrm{C}}_{\varphi }^{\text{train}}$ values ranging from $\sim 3.0\lambda$ to $\sim 0.5\lambda$, where $C_{\varphi }^{\text{test}}=C_{\varphi }^{\text{train}}$. The performance in each case was evaluated using 10,000 MNIST test images with PCC, diffraction efficiency, PSNR, and FOM metrics. The two-star markers in (d) represent the diffractive models with $C_{\varphi }^{\text{train}}=0.5\lambda$ and $C_{\varphi }^{\text{train}}=1.0\lambda$, both of which were trained using a high-resolution image dataset composed of random intensity patterns. (e) Examples of the blind testing results with different $C_{\varphi }^{\text{train}}$ values ranging from $\sim 3.0\lambda$ to $\sim 0.5\lambda$. The first three rows display the input amplitude objects, forward output images, and backward output images. For comparison, the last row shows the backward output images with different intensity ranges. Images with the same-colored frame share the same intensity range.

According to the FOM values presented in Fig. 4(d) and the visualizations in Fig. 4(e), partially coherent diffractive processors exhibit a very good unidirectional imaging performance ($\mathrm{FOM}\ge 4$) when trained with a larger correlation length of $C_{\varphi }^{\text{train}}\ge 1.5\lambda$, enabling unidirectional imaging with a large diffraction efficiency in the forward direction while suppressing image formation with a significantly reduced diffraction efficiency in the opposite direction. However, for diffractive processors trained with $C_{\varphi }^{\text{train}}<1.5\lambda$, the unidirectional imaging performance appears to diminish. Specifically, both the forward and backward output diffraction efficiencies of these diffractive models trained with $C_{\varphi }^{\text{train}}<1.5\lambda$ fall below 1%, accompanied by a reduced FOM of $\sim 1$ to 2 [Fig. 4(d)]. When illuminated with partially coherent light with shorter correlation lengths, images of the input patterns can be observed in both the forward and backward outputs.

To further explore the design characteristics of diffractive unidirectional imagers trained under different correlation lengths ranging from $0.5\lambda$ to $3\lambda$, Fig. 5 illustrates the phase profiles of the optimized diffractive layers of each design. We observe that the central parts of the diffractive layers trained using $C_{\varphi }^{\text{train}}=0.5\lambda$ and $C_{\varphi }^{\text{train}}=1.0\lambda$ are relatively flat, which indicates poor imaging performance, since these central parts are crucial for image formation. In contrast, the diffractive layers trained with larger correlation lengths, $C_{\varphi }^{\text{train}}\ge 1.5\lambda$, exhibit a completely different topology in each diffractive layer, indicating better learning/convergence and more effective utilization of the independent degrees of freedom at each diffractive layer, which is at the heart of better unidirectional imaging performance achieved for these diffractive models trained with $C_{\varphi }^{\text{train}}\ge 1.5\lambda$, as also shown in Fig. 4.

Fig. 5

Optimized phase profiles of the diffractive layers of different unidirectional imagers trained with varying $C_{\varphi }^{\text{train}}$ - from $\sim 0.5\lambda$ to $\sim 3.0\lambda$.

So far, we used the same level of partial spatial coherence during both the training and blind testing stages, i.e., $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}$. Next, we explored the generalization performance of a unidirectional diffractive imager under different levels of partial coherence during the blind testing stage (see Fig. 6). Each line in Fig. 6 depicts the performance of a unidirectional imager design (trained using a specific $C_{\varphi }^{\text{train}}$) with respect to varying $C_{\varphi }^{\text{test}}$ values. These findings support our previous observations, revealing that the unidirectional diffractive imagers trained with $C_{\varphi }^{\text{train}}\ge 1.5\lambda$ perform well, and these diffractive designs do not necessarily overfit to a specific $C_{\varphi }^{\text{train}}$ value, exhibiting improved FOM as long as $C_{\varphi }^{\text{test}}\ge 1.5\lambda$; also see Figs. S3–S5 in the Supplementary Material for some blind testing examples further supporting these conclusions.

Fig. 6

The generalization performance of unidirectional diffractive imagers across various $C_{\varphi }^{\text{test}}$ values, ranging from $\sim 0.5\lambda$ to $\sim 3\lambda$, despite being trained with a specific $C_{\varphi }^{\text{train}}$.

In Fig. 6 and Figs. S4 and S5 in the Supplementary Material, we also observe that none of these diffractive designs achieves a decent unidirectional imaging FOM when tested with $C_{\varphi }^{\text{test}}=0.5\lambda$ and $C_{\varphi }^{\text{test}}=1.0\lambda$, i.e., when the phase correlation length of the illumination beam approaches the diffraction limit of light in air. This poor performance of the unidirectional imager designs reported in Fig. 6 with $C_{\varphi }^{\text{test}}<C_{\varphi }^{\text{train}}\ge 1.5\lambda$ is due to the fact the diffractive layers shown in Fig. 5 for $C_{\varphi }^{\text{train}}\ge 1.5\lambda$ overfitted to the relatively larger spatial coherence diameter of the illumination, failing external generalization and unidirectional imaging for $C_{\varphi }^{\text{test}}=0.5\lambda$ and $C_{\varphi }^{\text{test}}=1.0\lambda$. However, the failure of the diffractive designs with $C_{\varphi }^{\text{test}}=C_{\varphi }^{\text{train}}=0.5\lambda$ and $C_{\varphi }^{\text{test}}=C_{\varphi }^{\text{train}}=1\lambda$ can be attributed to the lower spatial resolution and relative sparsity of our training dataset, failing to cover phase correlation lengths closer to the diffraction limit of light. To better shed light on this, we trained two new diffractive unidirectional imagers ($C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=0.5\lambda$ and $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=1.0\lambda$) with higher resolution training image datasets featuring random intensity patterns (see Figs. S6 and S7 in the Supplementary Material as well as Sec. 2 for details). Figure S6(e) in the Supplementary Material reveals that the diffractive layers of this new unidirectional imager design ($C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=0.5\lambda$) better utilize the independent degrees of freedom at each layer, and avoid relatively smooth large regions at the center of each layer, which indicates a better optimization of the unidirectional imager. This improved performance is also evident in the comparisons provided in Figs. S6(a)–S6(d), Figs. S7(a)–S7(d), and Fig. 4(e) in the Supplementary Material, where the FOM of unidirectional imaging increased to 3.32 and 3.75, respectively, revealing improvements of $>2.2$-fold compared to our earlier designs with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=0.5\lambda$ and $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=1.0\lambda$. Figures S6 and S7 in the Supplementary Material also report unidirectional imaging metrics of these designs, i.e., PCC, diffraction efficiency, and PSNR, which are calculated using 10,000 random test images, further supporting the improved performance of these new designs. These analyses underscore the crucial role of the training image dataset in the performance of a diffractive unidirectional imager, especially for a phase correlation length of $\le \lambda$.

Fig. 7

Image dataset external generalization for the unidirectional imager with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$. Blind testing results with EMNIST letters (a) and customized gratings (b), both of which were never used during training.

3.4.

External Generalization of Partially Coherent Unidirectional Diffractive Imager Designs to New Image Datasets

Next, we showcase the external generalization capabilities of partially coherent diffractive unidirectional imager designs to other datasets that had never been used before. For this analysis, we used the EMNIST dataset³⁰ containing images of handwritten English letters and a customized image dataset featuring various types of gratings. Both of these were never used during the training, which only used handwritten digits. These external generalization test results shown in Fig. 7 are obtained using the unidirectional imager design with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$. The first three rows in Figs. 7(a) and 7(b) depict the input amplitude objects, the forward outputs, and the backward outputs, respectively, all using the same intensity range. The last row shows the backward output images with increased image contrast, clearly confirming the unidirectional imaging capability and the successful external generalization of the diffractive design to unseen input images from different datasets.

We also quantified the spatial resolution of this partially coherent unidirectional imager ($C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$) using various resolution test targets with different linewidths that were previously unseen (see Fig. 8). The minimum resolvable linewidth (period) by this unidirectional imager design was found to be $\sim 1\lambda$ ($\sim 2\lambda$), as indicated by the results in Fig. 8. These results further validate the successful external generalization of the unidirectional imager design for general-purpose imaging operation exclusively in the forward direction, despite being trained solely using the MNIST image dataset. Additionally, we calculated the average gradient of the image cross sections to estimate the forward and backward point-spread functions (PSFs) of the unidirectional imager, shown in Fig. S8 in the Supplementary Material. Each line displayed in Fig. S8 in the Supplementary Material was averaged over five evenly spaced cross sections (along the $x$ and $y$ directions) within the resolution test targets derived from the diffractive output images in Fig. 8. The sharp gradient peaks in the forward direction and the flat gradients in the backward direction highlight the asymmetric visual information processing of our unidirectional imager, as desired, demonstrating a decent resolution in the forward imaging direction while effectively blocking image information in the backward direction. Note that such gratings or resolution test targets were never seen by the unidirectional imager during the training process, and these presented results reflect the external generalization performance. The resulting imaging quality could be further improved by incorporating higher-resolution targets into the optimization process.

Fig. 8

Spatial resolution analysis for a unidirectional diffractive imager design with $C_{\varphi }^{\text{train}}=C_{\varphi }^{\text{test}}=2.5\lambda$.