Open Access
1 May 2009 Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle
Silvia Santos, Kengyeh K. Chu, Daryl Lim, Nenad Bozinovic, Timothy N. Ford, Claire Hourtoule, Aaron C. Bartoo, Satish K. Singh, Jerome Mertz
Author Affiliations +
Abstract
We present an endomicroscope apparatus that exhibits out-of-focus background rejection based on wide-field illumination through a flexible imaging fiber bundle. Our technique, called HiLo microscopy, involves acquiring two images, one with grid-pattern illumination and another with standard uniform illumination. An evaluation of the image contrast with grid-pattern illumination provides an optically sectioned image with low resolution. This is complemented with high-resolution information from the uniform illumination image, leading to a full-resolution image that is optically sectioned. HiLo endomicroscope movies are presented of fluorescently labeled rat colonic mucosa.

The development of a simple, robust high-resolution fluorescence endomicroscope is driven by preclinical and clinical needs.1 Standard wide-field techniques are hampered by their inability to to reject out-of-focus background, generally leading to low signal contrast. Strategies to reduce out-of-focus background have been based on confocal detection2, 3, 4, 5, 6 or two-photon excitation,7, 8 both requiring some sort of scanning mechanism. Alternatively, out-of-focus background can be rejected in a nonscanning wide-field endoscope by use of structured illumination microscopy (SIM),9 which we have implemented with a flexible fiber bundle.10 While SIM is effective at optical sectioning, we have found that it is highly susceptible to sample motion, the difficulty being that high-resolution image information in SIM is distributed over a series of at least three raw images, meaning that any misregistration between the raw images leads to artifacts in the final processed SIM image. Recently, we have developed a novel hybrid-illumination technique to address this problem.11 In this technique, two raw images are required, only one containing high-resolution information and the other containing low-resolution information. Hence, the name of our new imaging technique: HiLo microscopy.

The two raw images required for HiLo microscopy are based respectively on uniform and nonuniform (or structured) illumination. In our initial implementation of HiLo microscopy, the nonuniform illumination was obtained with laser speckle.11 However, HiLo microscopy is more general than this and can be implemented with any type of nonuniform illumination. In particular, we demonstrate here the implementation of HiLo endomicroscopy with nonuniform illumination in the form of a grid pattern, of the same type as used in SIM. Indeed, our HiLo endomicroscope setup is identical to our previous SIM endomicroscope setup (see Fig. 1 ), except that for HiLo, the spatial light modulator now toggles between two illumination patterns, grid and uniform, whereas for SIM, it sequentially produced three grid patterns of incrementing phase.10

Fig. 1

(a) HiLo endomicroscope setup: an expanded laser beam (Cobolt Calypso, λ=491nm ) is directed onto a spatial light modulator (Holoeye LC-R 768) to create grid and uniform illumination patterns, which are then projected into an imaging fiber bundle ( 600-μm useful diameter; 30,000 fibers; 1.9-μm core diameters; 3.3-μm core separations) equiped with a distal micro-objective (Mauna Kea Technologies: NA=0.8 water, working distance=60μm , field of view=240μm ). The resultant fluorescence is isolated with a dichroic and emission filter (Chroma) and recorded with a CCD camera (QImaging Retiga). Raw images of fluorescently labeled lens-paper fibers with (b) grid and (c) uniform illumination.

030502_1_005903jbo1.jpg

The principle of HiLo microscopy was described in Ref. 11. In brief, a final optically sectioned HiLo image is constructed from the fusion of complementary in-focus high- and low-frequency image components. High-frequency components in the uniform illumination image are inherently in focus and are extracted with a high-pass filter. In-focus low-frequency components, on the other hand, must be extracted in a more complicated manner, since the simple application of a low-pass filter to the uniform illumination image does not reject out-of-focus background. To reject low-frequency out-of-focus background, we evaluate the local contrast in the fluorescence image obtained with nonuniform illumination. This local contrast is higher for in-focus image components than for out-of-focus components and hence is axially resolved. A multiplication of the local image contrast with the original uniform illumination image then provides an optically sectioned image, although at low resolution. A fusion of this in-focus low-resolution image with the complementary high-resolution image (inherently in focus) then leads to a full-resolution image that is axially resolved over all spatial frequencies within the microscope passband.

A key step in HiLo microscopy is the extraction of local image contrast from the nonuniform illumination image. In our previous implementation where the nonuniform illumination consisted of laser speckle, this local image contrast was evaluated by calculating the standard deviation of the nonuniform illumination image intensity over local, coarse-grained resolution areas. In our current implementation with a grid pattern, we will adopt a slightly different, although essentially equivalent, approach based on single-sideband demodulation. To understand this approach, let us phenomenologically decompose our uniform illumination image Iu(ρ) into in-focus and out-of-focus components. That is, we write

Eq. 1

Iu(ρ)=Iin(ρ)+Iout(ρ),
where ρ={x,y} are spatial coordinates in the image plane. Our final goal is to isolate Iin(ρ) .

The nonuniform image can be decomposed similarly into

Eq. 2

In(ρ)=Iin(ρ)[1+Msin(κgx+φ)]+Iout(ρ),
where the (imaged) grid illumination is modeled as a sinusoidal pattern of spatial frequency κg in the x direction, with arbitrary phase φ and (imaged) modulation contrast M . Note that only the in-focus image component appears to be modulated, whereas the out-of-focus component does not, precisely because the latter is out of focus.

In(ρ) and Iu(ρ) are the two raw images required for HiLo microscopy. The ratio R(ρ)=In(ρ)Iu(ρ) of these two images leads to

Eq. 3

R(ρ)=1+C(ρ)Msin(κgx+φ),
where C(ρ)=Iin(ρ)[Iin(ρ)+Iout(ρ)] is the local image contrast that we have set out to derive. The Fourier transform of R(ρ) —namely, R(κ) —is illustrated in Fig. 2, where we observe that C(κ) [the Fourier transform of C(ρ) ] and its complex conjugate reside in sidebands centered at ±κg . The more highly contrasted the image, the taller and narrower these sidebands become. While several techniques may be used to extract C(κ) from R(κ) , we adopt the standard technique of single-sideband demodulation. In the frequency domain, this corresponds to isolating only a single sideband R+(κ) by applying a one-sided high-pass filter to R(κ) (dashed line in Fig. 2), followed by an inverse Fourier transform to retrieve R+(ρ) . Provided the sidebands are well separated from one another (i.e., they do not overlap), then the local image contrast is given by C(ρ)=[R+(ρ)R+*(ρ)]12M . From C(ρ) , derived in this manner, and the uniform illumination image Iu(ρ) , we can then infer Iin(ρ)=C(ρ)Iu(ρ) . That is, we can extract Iin(ρ) from Iu(ρ) , thereby obtaining an optically sectioned image containing only in-focus contributions.

Fig. 2

Schematic of R(κ) . The right sideband R+(κ) is isolated from the left sideband and from the zero-frequency delta function by applying a one-sided, high-pass filter to R(κ) (dashed line). Note: The exact cutoff response of the filter function can be chosen somewhat arbitrarily.

030502_1_005903jbo2.jpg

There are two difficulties with the preceding procedure. First, the sidebands R(κ) and R+(κ) may be so wide as to overlap. This problem can be alleviated by choosing a one-sided, high-pass filter cutoff profile that helps suppress the overlap region. The second problem is that, in general, M is not known a priori. We thus define a new parameter Isu(ρ)=[R+(ρ)R+*(ρ)]12Iu(ρ) that is independent of M . Moreover, we purposefully restrict Isu(ρ) to spatial frequencies smaller than κg by applying a low-pass filter to Isu(ρ) , with user-defined cutoff frequency κcκg , obtaining Ilp(ρ)=LP[Isu(ρ)] . In practice, LP[Isu(ρ)] is performed by convolving Isu(ρ) with a square window of size 2πκg (or integral multiple thereof, to minimize the possibility of aliasing). In addition to confining Isu(ρ) to a well-defined bandwidth, such filtering helps suppress potential artifacts arising, for example, from a nonperfectly sinusoidal illumination pattern.

Finally, the low-resolution image Ilp(ρ) is combined with complementary high-resolution information Ihp(ρ) obtained by applying a high-pass filter directly to the uniform illumination image, such that Ihp(ρ)=HP[Iu(ρ)]=Iu(ρ)LP[Iu(ρ)] . The final processed HiLo image is given by

Eq. 4

Ihilo(ρ)=ηIlp(ρ)+Ihp(ρ),
where η is a scaling factor introduced to ensure a seamless transition of the frequency content of Ihilo(ρ) across the cutoff frequency κc . [In effect, the introduction of η compensates for the fact that M is unknown—see Ref. 11 for an explanation of how η is derived from Ilp(ρ) and Ihp(ρ) , based on their complementary nature.] We emphasize that Ihilo(ρ) is axially resolved over all spatial frequencies, both high and low, within the endomicroscope passband and therefore constitutes an optically sectioned image. Moreover, it provides a faithful rendition of the sample since, to high accuracy, Ihilo(ρ) is directly proportional to the in-focus fluorophore concentration in the sample.

Figure 3 provides comparisons of standard wide-field and HiLo endomicroscope images through a fiber bundle. It should be noted that a longer grid period leads to a stronger imaged modulation depth M —however, at the expense of weaker HiLo optical sectioning capacity. A grid period of 30μm was found to provide a reasonable compromise between grid pattern contrast and HiLo optical sectioning capacity. A rough measure of this sectioning capacity can be inferred from a measurement of the detected signal strength from a uniform fluorescent half-space whose interface is scanned through the focal plane. A comparison of the signal strengths acquired with wide-field and HiLo endomicroscopy is illustrated in Fig. 3d, where we observe that the HiLo signal strength decays much more precipitously than the wide-field signal strength. The corresponding HiLo axial resolution for a laterally uniform plane is inferred from the derivative of the HiLo signal strength, and is found, in this case, to be about 30-μm FWHM. Note that for a laterally uniform sample, Ihp(ρ) vanishes, and the HiLo image is comprised solely of Ilp(ρ) .

Fig. 3

Intermediate (a) Ilp(ρ) and (b) Ihp(ρ) images obtained from raw images in Fig. 1, and (c) final HiLo image. [Note: Panel (b) contains both positive and negative values.) (d) Demonstration of HiLo sectioning capacity using axially scanned fluorescent half-space (Chroma plastic slide—see inset.)

030502_1_005903jbo3.jpg

Finally, a comparison of wide-field and HiLo endomicroscopic imaging of colon tissue in motion is illustrated in the multimedia movies ( Videos 1 and Videos 2 ). Note the absence of a residual grid pattern or other motion-related artifacts. Our net HiLo imaging acquisition rate was about 2Hz (i.e., 250ms per raw image). This rate was software limited and will be improved in future versions of our apparatus. Our imaging resolution is about 2.6μm , limited by the Nyquist frequency associated with the (magnified) fiber core quasiperiodicity (see Ref. 10).

Video 1

Wide-field movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (MPG, 3.2MPG ).

030502_1_005903jbov1.jpg
10.1117/1.3130266.1

Video 2

HiLo movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (acquired with same grid period as in Fig. 3) (MPG, 3.2MG ).

030502_1_005903jbov2.jpg
10.1117/1.3130266.2

In conclusion, we have successfully demonstrated the capacity of HiLo microscopy to provide optically sectioned imaging of fluorescently labeled colon tissue through a flexible optical fiber bundle. We anticipate that this will open new possibilities in high-resolution fluorescence endomicroscopy.

Acknowledgments

This work was partially funded by the NIH (R21 EB007338).

References

1. 

A. Hoffman, M. Goetz, M. Vieth, P. R. Galle, M. F. Neurath, and R. Kiesslich, “Confocal laser endomicroscopy: technical status and current indications,” Endoscopy, 38 1275 –1283 (2006). https://doi.org/10.1055/s-2006-944813 0013-726X Google Scholar

2. 

A. R. Rouse and A. F. Gmitro, “Multispectral imaging with a confocal microendoscope,” Opt. Lett., 25 1708 –1710 (2000). https://doi.org/10.1364/OL.25.001708 0146-9592 Google Scholar

3. 

L. D. Swindle, S. G. Thomas, M. Freeman, and P. M. Delaney, “View of normal human skin in vivo as observed using fluorescent fiber-optics confocal microscopic imaging,” J. Invest. Dermatol., 121 706 –712 (2003). https://doi.org/10.1046/j.1523-1747.2003.12477.x 0022-202X Google Scholar

4. 

F. Jean, G. Bourg-Heckly, and B. Viellerobe, “Fibered confocal spectroscopy and multicolor imaging system for in vivo fluorescence analysis,” Opt. Express, 15 4008 –4017 (2007). https://doi.org/10.1364/OE.15.004008 1094-4087 Google Scholar

5. 

H.-J. Shin, M. C. Pierce, D. Lee, H. Ra, O. Solgaard, and R. Richards-Kortum, “Fiber-optic confocal microscope using MEMS scanner and miniature objective lens,” Opt. Express, 15 9113 –9122 (2007). https://doi.org/10.1364/OE.15.009113 1094-4087 Google Scholar

6. 

J. T. C. Liu, M. J. Mandella, H. Ra, L. K. Wong, O. Solgaard, G. S. Kino, W. Piyawattanametha, C. H. Contag, and T. D. Wang, “Miniature near-infrared dual-axes confocal microscope utilizing two-dimensional microelectromechanical systems scanner,” Opt. Lett., 32 256 –258 (2007). https://doi.org/10.1364/OL.32.000256 0146-9592 Google Scholar

7. 

B. Flusberg, E. Cocker, W. Piyawattanametha, J. Jung, E. Cheung, and M. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods, 2 941 –950 (2005). https://doi.org/10.1038/nmeth820 1548-7091 Google Scholar

8. 

P. Kim, M. Puoris’haag, D. Coté, C. P. Lin, and S. H. Yun, “In vivo confocal and multiphoton microendoscopy,” J. Biomed. Opt., 13 010501 (2008). https://doi.org/10.1117/1.2839043 1083-3668 Google Scholar

9. 

D. Karadaglic, R. Juškaitis, and T. Wilson, “Confocal endoscopy via Structured Illumination,” Scanning, 24 301 –304 (2002). 0161-0457 Google Scholar

10. 

N. Bozinovic, C. Ventalon, T. Ford, and J. Mertz, “Fluorescence endomicroscopy with structured illumination,” Opt. Express, 16 8016 –8025 (2008). https://doi.org/10.1364/OE.16.008016 1094-4087 Google Scholar

11. 

D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett., 33 1819 –1821 (2008). https://doi.org/10.1364/OL.33.001819 0146-9592 Google Scholar
©(2009) Society of Photo-Optical Instrumentation Engineers (SPIE)
Silvia Santos, Kengyeh K. Chu, Daryl Lim, Nenad Bozinovic, Timothy N. Ford, Claire Hourtoule, Aaron C. Bartoo, Satish K. Singh, and Jerome Mertz "Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle," Journal of Biomedical Optics 14(3), 030502 (1 May 2009). https://doi.org/10.1117/1.3130266
Published: 1 May 2009
Lens.org Logo
CITATIONS
Cited by 83 scholarly publications and 1 patent.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Image fusion

Microscopy

Luminescence

Endomicroscopy

Linear filtering

Image resolution

Image processing

Back to Top