We present the Fourier Light field Camera Array Microscope (FL-CAM) for high-throughput, single-snapshot 3D imaging of mesoscale samples, particularly freely moving model organisms. The FL-CAM substitutes the micro-lens array of typical light field systems for a synchronized array of 48 independent imaging systems. The FL-CAM can capture multi-perspective images over a 3.1 cm x 4.1 cm field of view with 38o angular range at up to 200 frames per second and uses a physics-supervised machine learning algorithm, which accounts for the unique distortion patterns created by the cascade of lenses, to achieve 3D visualization.
We introduce an innovative MCAM architecture using a 6x8 array of 48 lenses and sensors for simultaneous 0.624 gigapixel imaging within a few centimeters, delivering near-cellular resolution. This enables 3D video recording and radiometric fluorescence imaging of organisms using stereoscopic capture and appropriate filters. Such a feature proves advantageous when conducting combined investigations into organism behavior and functional fluorescence measurements. Moreover, the MCAM is equipped to perform birefringent imaging by incorporating suitable polarizers. We demonstrate the multimodal imaging capacity of this system using a variety of specimens, notably Drosophila, and zebrafish.
We present a high-resolution, wide-field of view (FOV) computational microscope that employs an array of image sensors with gaps between them and a diffractive optical element (DOE) in the Fourier plane. The sensor array consists of a 6 x 8 array of 13-megapixel sensors (total ~0.6 gigapixels), spanning a 5 cm by 6.6 cm region with a ~22% fill factor. To fill in the inter-sensor gaps without scanning, we introduce a DOE at the pupil that generates a distributed PSF, allowing us to multiplex information from the missing ~78% of the total area into the sensing regions. Our large-scale reconstruction algorithm demixes the superimposed information, resulting in a >4x expanded FOV. Our approach can enable multi-gigapixel imaging in a single snapshot.
KEYWORDS: Organisms, Microscopes, Tomography, Algorithm development, 3D modeling, Detection and tracking algorithms, 3D tracking, 3D image processing, Reconstruction algorithms, Muscles
It is challenging to study behavior of and track freely-moving model organisms using conventional 3D microscopy techniques. To overcome motion artifacts and prevent the organism from leaving the field of view (FOV), existing techniques require paralyzing or otherwise immobilizing the organism. Here, we demonstrate hemispheric Fourier light field tomography, featuring a parabolic objective that enables synchronized multi-view fluorescence imaging over ~2pi steradians at up to 120 fps and across multi-millimeter 3D FOVs. Our method is not only able to track the 6D pose of freely-moving zebrafish and fruit fly larvae, but also other properties such as heartbeat, fin motion, jaw motion, and muscle contractions. We also demonstrate simultaneous multi-organism imaging.
In fields such as virtual/augmented reality, robotic vision, facial recognition, and biomedical imaging, the demand for accurate, fast coherent 3D surface imaging technology is increasing. However, current technologies like FMCW LiDAR have limitations in terms of low spectral bandwidth efficiency and high cost. We demonstrate a novel approach using synthetic wavelength phase unwrapping and line-scan off-axis holography. Our system achieved fast 3D surface imaging with a non-ambiguous depth range of 22.3mm. We validated system performance using a 3D-printed test target and a U.S. copper penny.
We report tensorial tomographic Fourier ptychography (T2oFu), a nonscanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier ptychography, a quantitative phase imaging technique, T2oFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equipped with an LED matrix, a polarization generator, and a polarization-sensitive camera. Permittivity tensors of anisotropic samples are computationally recovered from polarized intensity measurements across three dimensions. We demonstrate T2oFu’s efficiency through volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, as well as tissue samples from muscle fibers and diseased heart tissue. Our reconstructions of healthy muscle fibers reveal their 3D fine-filament structures with consistent orientations. Additionally, we demonstrate reconstructions of a heart tissue sample that carries important polarization information for detecting cardiac amyloidosis.
“Anyone who uses a microscope has likely noticed the limitation of the trade-off between the field of view and the resolution”. To acquire highly resolved images at large fields of view, existing techniques typically record sequential images at different positions and then digitally stitch composite images. There are alternatives to this mechanical scanning procedure, such as structured illumination microscopy or Fourier ptychography that record sequential images at varying illuminations prevent mechanical scanning for high-resolution image composites. However, all of these approaches require sequential images and thus compromise speed, temporal resolution and experimental throughput. Here we present the Multi-Camera Array Microscope (MCAM), which is a microscope system that utilizes an array of many synchronized cameras, each with an individual imaging lens, for simultaneous image capture. The MCAM enables enhanced imaging capabilities and novel applications in various scientific and medical fields, by combining the images acquired from each individual camera-lens pair.
We present a high-throughput computational imaging system capable of performing dense, volumetric fluorescence imaging of freely moving organisms at up to 120 volumes per second. Our method, termed 2pi Fourier light field tomography (2pi-FLIFT), consists of a planar array of 54 cameras and a parabolic mirror serving as the primary objective that allows for synchronized multi-view video capture over ~2pi steradians. 2pi-FLIFT features a novel 3D reconstruction algorithm that recovers both the 3D fluorescence distribution and attenuation map of dynamic samples. We demonstrate 2pi-FLIFT on important, freely moving model organisms, such as zebrafish and fruit fly larvae.
Conventional microscopy limits how much information we can capture about microscopic specimens. In particular, there is a tradeoff between field of view (FOV) and resolution. Here, we present a new parallelized microscope that can image up to 16 gigapixels over wide FOVs at micrometer resolutions. Our multi-camera array microscope (MCAM) consists of 48 micro-cameras, packed closely together to directly image different areas in parallel. We will demonstrate 2D and 3D brightfield, differential phase contrast (DPC), and fluorescence imaging with various specimens.
Well-known limitations of optical coherence tomography (OCT) include deleterious speckle noise and relatively poor lateral resolution (typically >10 μm) due the tradeoff between lateral resolution and depth of focus. To address these limitations, we present 3D optical coherence refraction tomography (OCRT), which computationally combines 3D volumes from two rotational axes to form a 3D reconstruction with substantially reduced speckle noise and enhanced lateral resolution. Our approach features a parabolic mirror as the objective, which enables multi-view OCT volume acquisition over up to ±75° without moving the sample. We demonstrate 3D OCRT on a phantom sample and several biological samples, revealing new structures that are missed in conventional OCT.
FMCW depth imaging is a coherent 3D imaging modality analogous to SS-OCT. Due to constraints of mechanical steering mirrors, meter-scale FMCW depth cameras typically suffer from low data rates (<1Hz 3D map rate). Here, we describe the design and construction of a high-speed FMCW depth camera that employs a grating for beam steering and a telescope for angular FOV magnification. Our camera produces 3D depth maps at 33Hz, each consisting of 475x500 pixels, spanning a depth range of 32.8cm with sub-millimeter depth localization accuracy. Our FMCW depth camera is suitable for room-scale real-time 3D imaging applications, particularly computer vision applications.
Microscopic imaging of anisotropic samples has many important applications in cytopathology. The endogenous contrast from the polarization properties of a specimen, such as its birefringence, provides valuable diagnostic information for several deadly diseases, including cardiac amyloidosis and squamous cell carcinoma, for example. In the past, polarized light microscopy (PLM) has been widely used as a diagnostic tool during the clinical review. However, in analogy with the standard microscope, the PLM typically has a restricted spatial-bandwidth product (SBP). As a consequence, one can either image a large area with low resolution or see the details of a very small area of the sample at the resolutions required for accurate analysis. To address the SBP issue of the PLM, we propose a computational microscopy method, termed vectorial Fourier ptychography, to illuminate the specimen with polarized light from different angles and detects different polarization states of the diffracted light. By illuminating a specimen with plane waves from different angles, our vectorial Fourier ptychography method effectively modulates the high-spatial-frequency components of the specimen into lower frequencies that can be detected by the optical system. With a Jones calculus-based forward model and a second-order phase retrieval method, we can reconstruct high-resolution, wide field-of-view(FOV) amplitude, phase, birefringence, retardance, and diattenuation of the specimen. To assess the reconstruction accuracy of our method, we imaged polystyrene beads submerged in immersion oils of different refractive index, as well as monosodium urate crystals. Further, To validate the diattenuation reconstruction accuracy, we reconstruct a USAF resolution test chart with a half blocked by a linear polarizer. These experiments confirm quantitatively accurate reconstruction results with a 1.25 um full-pitch resolution over a FOV of 6.6 x 4.4 mm^2, which is 5 times higher than the native (brightfield) resolution of the non-computational optical system. Finally, we demonstrate our technique by producing high SBP polarization images of several anisotropic biologic samples, includes collagen tissue, congo red stained cardiac tissue, and a bean root sample.
We present a gigapixel-scale multi-aperture microscope capable of measuring a sample’s 3D height profile over multi-centimeter-scale fields of view with a series of single synchronized camera snapshots. Exploiting the overlap redundancy in our multi-aperture camera array microscope, we developed a novel, end-to-end photogrammetric reconstruction algorithm that simultaneously calibrates the cameras’ 3D positions and poses, stitches the acquired images, and generates a coregistered, pixel-wise 3D height map of the sample. Our work opens the door to video-rate 3D monitoring of dynamic scenes at micrometer-scale resolutions and centimeter-scale fields of view.
More people die from melanoma after a stage I diagnosis than after a stage IV diagnosis, because the tools available to clinicians do not readily identify which early-stage cancers will be aggressive. Near-infrared pump-probe microscopy detects fundamental differences in melanin structure between benign human moles and melanoma and also correlates with metastatic potential. However, the biological mechanisms of these changes have been difficult to quantify, as many different mechanisms can contribute to the pump-probe signal. We use model systems (sepia, squid, and synthetic eumelanin), cellular uptake studies, and a range of pump and probe wavelengths to demonstrate that the clinically observed effects come from alterations of the aggregated mode from “thick oligomer stacks” to “thin oligomer stacks” (due to changes in monomer composition) and (predominantly) deaggregation of the assembled melanin structure. This provides the opportunity to use pump-probe microscopy for the detection and study of melanin-associated diseases.
Optical coherence tomography (OCT) enables non-invasive, high-resolution, tomographic imaging of biological tissues by leveraging principles of low coherence interferometry; however, OCT lacks molecular specificity. Spectroscopic OCT (SOCT) overcomes this limitation by providing depth-resolved spectroscopic signatures of chromophores, but SOCT has been limited to a couple of endogenous molecules, namely hemoglobin and melanin. Stimulated Raman scattering, on the other hand, can provide highly specific molecular information of many endogenous species, but lacks the spatial and spectral multiplexing capabilities of SOCT. In this work we integrate the two methods, SRS and SOCT, to enable simultaneously multiplexed spatial and spectral imaging with sensitivity to many endogenous biochemical species that play an important role in biology and medicine. The method, termed SRS-SOCT, has the potential to achieve fast, volumetric, and highly sensitive label-free molecular imaging, which would be valuable for many applications. We demonstrate the approach by imaging excised human adipose tissue and detecting the lipids’ Raman signatures in the high-wavenumber region. Details of this method along with validations and results will be presented.
One critical barrier to the robust study of cilia-driven fluid flow in developmental biology is a lack of methods for acquiring three-dimensional (3D) images of three vector component (3C) measurements of flow velocities. A 3D3C map of cilia-driven fluid flow quantifies the flow speed along three axes (e.g. three Cartesian vector components v_x, v_y, v_z) at each point in 3D space. 3D3C quantification is important because cilia-driven fluid flow is not amenable to simplifying assumptions (e.g. parabolic flow profile. Such quantification may enable systematically detailed characterization of performance using shear force and power dissipation metrics derived from 3D3C flow velocity fields.
We report our OCT-based results in developing methods for the 3D3C quantification of cilia-driven flow fields. First, we used custom scan protocols and reconstruction algorithms to synthesize 3D3C flow velocity fields from 2D2C fields generated using correlation-based methods (directional dynamic light scattering and digital particle image velocimetry). Xenopus results include flow driven by ciliated embryo skin and flow driven by ciliated ependymal cells in developing brain ventricles. Second, we developed a new approach to particle tracking velocimetry that generates 2D2.5C (2.5C: v_x,|v_y|,v_z) velocity fields from single-plane 2D image acquisitions. We demonstrated this particle streak velocimetry method in calibrated flow phantoms and in flow driven by ciliated Xenopus embryo skin. Additionally, we have preliminary results extending particle streak velocimetry to 3D3C in calibrated flow phantoms with ongoing work in Xenopus embryos.
We present a new method for 2.5 and 3 vector component velocimetry. We call this method particle streak velocimetry OCT (PSV-OCT). PSV-OCT generates two-dimensional, 2.5 vector component (v_x,|v_y|,v_z) cross-sectional maps of microscale flow velocity (e.g. biological cilia-driven fluid flow). The enabling insight is that a tracer particle in sparsely-seeded fluid flow traces out streaks in (x,z,t)-space. The streak orientations in x-t and z-t yield v_x and v_z, respectively. The in-plane (x-z plane) residence time yields the out-of-plane speed |v_y|. Vector component values are generated by fitting streaks to a model of image formation. We demonstrate cross-sectional estimation of (v_x,|v_y|,v_z) in two important animal models in ciliary biology: Xenopus embryos (tadpoles) and mouse trachea. Further, by incorporation the assumption of incompressible flow into the estimation process, we are able to generate 3 vector component (v_x,v_y,v_z) estimates in three spatial dimensions from 2.5 vector component measurements taken in parallel OCT planes in 3D space.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.