2.1.

Experimental Setup

The schematic diagram of the proposed x-ray and fluorescence imaging system is shown in Figs. 1(a) and 1(b). The two imaging modalities share the same optical path. An x-ray tube (Oxford Instruments Ultrabright, 50 kV, 0.8 mA) and a mercury lamp (Lumen Dynamics X-Cite Exacte, 200 W, 340 to 675 nm) are employed as the corresponding excitation source. The x-ray beam is conical, and the excitation light emitted from the mercury lamp is collimated. The sample is positioned horizontally and imaged on a digital micromirror device (DMD, X-digit XD-ED01N, 0.7 in., $768\times 1024$ micromirrors, pixel size $13.6\times 13.6{\mu \mathrm{m}}^2$) with a lens (Thorlabs AC254-035-A, focal length 35 mm). Structured detection is achieved by the micromirrors in DMD, turning on and off separately about the diagonal axis. And the modulated image is collected by a photomultipliers tube (Thorlabs PMM02, 300 to 800 nm) as the single-pixel detector.

Fig. 1

X-ray and fluorescence multimodality FSI system. (a) X-ray imaging configuration. (b) Fluorescence imaging configuration.

In x-ray imaging, the spatial modulation of x-ray is hard to realize due to the lack of effective modulator.²¹ Here, we place a CsI (Tl) scintillator plate (Hamamatsu Photonics J8734, effective area of $48\times 48{\mathrm{mm}}^2$, $150\text{-}\mu \mathrm{m}$-thick CsI scintillator) behind the sample. And the nonvisible x-ray containing the anatomical information can be converted to visible light and detected by the photomultipliers tube. The structured detection of x-ray image with the FSI architecture using DMD can be realized. To maximize the imaging field of view, the sample is placed along the diagonal of the scintillator plate, and the DMD chip is tilted for binary modulation. In this way, we can acquire the nonvisible x-ray and visible fluorescence images with only one visible light sensed single-pixel detector.

2.2.

Data Acquisition and Image Reconstruction

The principle of structured detection with three-step phase-shifting FSI architecture is shown in Fig. 2. Instead of sampling in the spatial domain, FSI acquires the Fourier spectrum of the object image.^22,23 For a certain Fourier coefficient $F(u,v)$ of an $M\times N$ pixels image $I$, three Fourier basis patterns $P_{Ø}(x,y,u,v)$ with initial phase $Ø$ ($Ø=\mathrm{0,2}\pi /3,4\pi /3$ rad) are needed for structured detection:

Fig. 2

Principle of structured detection with three-step phase-shifting FSI architecture.

$u,v$, and $x,y$ separately represent the 2D Cartesian coordinates in the Fourier and space domain. The patterns are binarized by Floyd–Steinberg dithering for fast FSI, and the corresponding responses $D_{Ø}$ can be shown as

2.3.

Animal Preparation

A C57BL/6 mouse (female, SPF, 4 weeks, 10 g) was subcutaneously inoculated with $5\times {10}^5$ B16 cells targeted by mCherry fluorescence protein. Once the tumor reached a volume of $100{\mathrm{mm}}^3$, the mouse was anesthetized, and in vivo imaging was carried out using our system. All animal experiments were performed according to the animal experiment guidelines of the Animal Experimentation Ethics Committee of Huazhong University of Science and Technology.

3.1.

System Performance Evaluation

To evaluate the performance of the proposed system, we measured the spatial resolution of x-ray imaging and the detection sensitivity as well as the imaging depth of fluorescence imaging. In the following experiments, we reconstruct the image with $96\times 128\mathrm{pixels}$. The pixel size in object space is $0.52\times 0.52{\mathrm{mm}}^2$, and the sampling ratio is 14% with circular sampling method applied in the Fourier domain. DMD refreshes at 20 Hz, and the acquisition time is 2 min for each image. No postprocessing methods were applied to the following results of our system.

As shown in Fig. 3(a), to estimate the spatial resolution of x-ray imaging, a metal wire with diameter of 0.4 mm was imaged. Reconstructed intensity along the line in Fig. 3(a) is shown in Fig. 3(b). Gaussian fitting to the profile of the metal wire indicates the spatial resolution of x-ray imaging is about 1.81 mm.

Fig. 3

The spatial resolution and fluorescence sensitivity evaluation of the proposed multimodality FSI system. (a) X-ray imaging result of a 0.4-mm-diameter metal wire, $96\times 128\mathrm{pixels}$. (b) Intensity profile with the corresponding Gaussian fit along the line indicated in (a), the full width at half maximum is 1.81 mm. (c) Fluorescence imaging result of five serially diluted rhodamine 6G solutions and pure water, from right to left, $96\times 128\mathrm{pixels}$. (d) Reconstructed intensity of the five rhodamine 6G solutions and pure water with $5\times 5\mathrm{pixels}$ averaging. Linear fit to the four detectable Rhodamine 6G solutions’ fluorescence intensity, $R^2=0.98$. Scale bars: 10 mm.

Then, we evaluated the detection sensitivity of fluorophore of the system in fluorescence modality. $200\text{-}\mu \mathrm{l}$ Rhodamine 6G solution with the serially diluted concentration of 11.86, 5.93, 2.97, 1.48, and 0.74 nmol/ml and pure water as the control group were imaged with the excitation light of 520 to 540 nm. The detected fluorescence ranges from 560 to 580 nm. As shown in Fig. 3(c), the pure water is placed leftmost with the concentration of rhodamine 6G solution reduced from right to left. We calculated the reconstructed intensities of different samples with $5\times 5\mathrm{pixels}$ averaging in Fig. 3(c). Linear fitting of the reconstructed fluorescence intensity with $R^2=0.98$ suggests that the minimum detectable concentration of rhodamine 6G solution is better than 1.48 nmol/ml.

Next, to demonstrate the imaging depth of fluorophore in fluorescence modality, the intralipid solution with 1 wt% was chosen as an optical phantom of biological tissue.²⁵ The optical property of the phantom can be characterized by the absorption coefficient ${\mu }_a=0.03{\mathrm{cm}}^{-1}$²⁶ and reduced scattering coefficient ${\mu }_s^{\prime }=12.3{\mathrm{cm}}^{-1}$.²⁷ Rhodamine 6G solution with the concentration of 11.86 nmol/ml was injected to a glass tube with diameter of 1 mm as the fluorescence probe. As shown in Figs. 4(a)–4(d), the glass tube was immersed at the depth of 2, 4, 6, and 8 mm in intralipid solution, respectively, the phantom test was carried out with the excitation light of 520 to 540 nm and detected fluorescence ranges from 560 to 580 nm. In Figs. 4(a) and 4(b), the glass tube can be clearly distinguished from the intralipid solution, so the imaging depth of the system in fluorescence modality can reach 4 mm.

Fig. 4

The imaging depth evaluation of the proposed multimodality FSI system using a phantom test. (a)–(d) The imaging results of a glass tube with diameter of 1 mm injected with rhodamine 6G solution and immersed at the depth of 2, 4, 6, and 8 mm in 1 wt% intralipid solution, $96\times 128\mathrm{pixels}$. Scale bars: 10 mm.

3.2.

In Vivo Multimodality Imaging of Tumor Bearing Mouse

To further examine the feasibility of the proposed system for small animal research, a C57BL/6 female mouse bearing tumor targeted with mCherry was imaged using our system. A commercial scientific complementary metal-oxide-semiconductor camera (Hamamatsu Photonics ORCA-Flash4.0 V3, $2048\times 2048\mathrm{pixels}$, pixel size: $6.5\times 6.5{\mu \mathrm{m}}^2$) was employed to validate our experimental results.

In fluorescence imaging modality, the fluorescence of the targeted tumor was excited with light from 560 to 580 nm and detected in the 600 to 620 nm range. The same experimental set up and parameters concerning the reconstructed image of our system were employed as mentioned in Sec. 3.1. The imaging results are shown in Fig. 5. The original FSI results are rotated to a specific angle for better comparison and framed by the red dotted line. Figures 5(a)–5(c) show the x-ray image, fluorescence image, and merged image, respectively. And Figs. 5(d) and 5(e) show the corresponding images captured with the sCMOS camera, respectively. When we acquired the images with an sCMOS camera, the results were resized to $165\times 165\mathrm{pixels}$ with pixel size of $0.52\times 0.52{\mathrm{mm}}^2$ in the object space. And the exposure time of the sCMOS camera is 5 s and 10 s for x-ray imaging and fluorescence imaging, respectively.

Fig. 5

Multimodality imaging results of a C57BL/6 mice bearing tumor targeted with mCherry fluorescence protein using the proposed FSI system (top row, rotated to a specific angle and framed by the red dotted line, $96\times 128\mathrm{pixels}$) and the commercial sCMOS camera (bottom row, $165\times 165\mathrm{pixels}$). (a)–(c) The x-ray, fluorescence, and merged images, respectively. (d) and (e) The corresponding images captured with the sCMOS camera. Scale bars: 10 mm.

As shown in Fig. 5(a), the x-ray imaging is able to provide the necessary anatomical information, including oral cavity, skeleton, and soft tissue. Figure 5(b) shows the molecular information of the small animal, which indicated the location of the tumor cell. And the merged image from the two modalities will provide the anatomical and molecular information simultaneously. The images acquired from the sCMOS camera confirm the results from our system.