SignificanceTo enable non-destructive longitudinal assessment of drug agents in intact tumor tissue without the use of disruptive probes, we have designed a label-free method to quantify the health of individual tumor cells in excised tumor tissue using multiphoton fluorescence lifetime imaging microscopy (MP-FLIM).AimUsing murine tumor fragments which preserve the native tumor microenvironment, we seek to demonstrate signals generated by the intrinsically fluorescent metabolic co-factors nicotinamide adenine dinucleotide phosphate [NAD(P)H] and flavin adenine dinucleotide (FAD) correlate with irreversible cascades leading to cell death.ApproachWe use MP-FLIM of NAD(P)H and FAD on tissues and confirm viability using standard apoptosis and live/dead (Caspase 3/7 and propidium iodide, respectively) assays.ResultsThrough a statistical approach, reproducible shifts in FLIM data, determined through phasor analysis, are shown to correlate with loss of cell viability. With this, we demonstrate that cell death achieved through either apoptosis/necrosis or necroptosis can be discriminated. In addition, specific responses to common chemotherapeutic treatment inducing cell death were detected.ConclusionsThese data demonstrate that MP-FLIM can detect and quantify cell viability without the use of potentially toxic dyes, thus enabling longitudinal multi-day studies assessing the effects of therapeutic agents on tumor fragments.
We aim to use the resolving power of near-infrared (NIR) fluorescence lifetime microscopy (FLIM) to provide information on the fluorescence decay behavior of NIR FRET donor probes, both in solution and in vitro, and assess their impact on in vivo macroscopic FLI FRET (MFLI FRET) tumor imaging. Utilizing HER2 mAbs, i.e., trastuzumab (TZM), labeled with AlexaFluor 700 (AF700), and HER2 positive cancer cell lines (AU565 and SKOV-3), we have documented significant impacts of IRF extraction methods and probe labeling schemes on FLIM analysis. Additionally, we have noted marked variation in the intracellular distribution of the HER2-TZM binding complexes, as well as in average endosomal lifetime measurements between cell lines. Herein, we discuss optimal methods for IRF extraction and generating NIR probes, as well as results from the newly optimized NIR FLIM FRET assay demonstrating variations in the average intracellular lifetime of TZM-AF700. Because fluorescence lifetime is impacted by environmental factors, such as pH, refraction, viscosity, and proximity to other molecules, these variations imply differences in the way TZM interacts with the endosomal microenvironment of these cell lines. We hypothesize that different HER2 positive cancer types exhibit variations in endosomal trafficking of the HER2-drug complex that play a key role in primary/acquired resistance to TZM.
Förster Resonance Energy Transfer (FRET) is a unique biophysical phenomenon that allows for energy transfer to occur between two light-sensitive molecules, e.g., a donor and an acceptor fluorophore. FRET has been leveraged in numerous biomedical applications to monitor molecular interactions at the nanoscale. Previously, we have developed a FRET-based nanometer-range proximity assay (2–10 nm) that measures receptor-ligand protein complexes during internalization and subsequent endocytic trafficking steps in vitro and in vivo. Recently, there has been great interest to perform FRET imaging in vivo, especially in the context of quantifying drug delivery in vivo.
KEYWORDS: Tumors, Fluorescence resonance energy transfer, In vivo imaging, Luminescence, Collagen, Resonance energy transfer, Receptors, Ovarian cancer, Near infrared, Multiplexing
Our goal is to accelerate pre-clinical drug discovery by developing novel imaging assays to screen and optimize the delivery of targeted anti-cancer drugs. Fluorescence lifetime imaging (FLI) Forster Resonance Energy Transfer (FRET) acts as a direct reporter of drug-target engagement in live mice carrying HER2-overexpressing tumor xenografts. We have established near-infrared (NIR) Macroscopy FLI FRET (MFLI-FRET) non-invasive imaging approach to measure drug-target engagement in deep tissues. We used trastuzumab (TZM), an anti-HER2 antibody clinical drug, as NIR-labeled FRET probes to assess quantitatively the role of tumor microenvironment on drug-target binding and penetration in tumor xenografts.
Reconstructions in 3D widefield Diffuse Optical Tomography (DOT) suffer from poor spatial resolution. Therefore, widefield DOT techniques benefit from incorporating structural priors from a complementary modality, such as the micro-CT. Unfortunately, traditional Laplacian-based methods to integrate the priors in the inverse problem are highly time-consuming. Therefore, we propose a Deep Neural Network based end-to-end inverse solver that combines features from AUTOMAP and Z-net and utilizes the micro-CT priors in the training stage. Initial in silico and experimental phantom results demonstrate that the proposed network accurately reconstructs, in 3D, the absorption contrast with a high resolution.
KEYWORDS: Monte Carlo methods, Data modeling, Luminescence, Computer simulations, Reflectivity, Sensors, In vivo imaging, Absorption, Diffuse optical tomography, 3D modeling
Significance: Deep learning (DL) models are being increasingly developed to map sensor data to the image domain directly. However, DL methodologies are data-driven and require large and diverse data sets to provide robust and accurate image formation performances. For research modalities such as 2D/3D diffuse optical imaging, the lack of large publicly available data sets and the wide variety of instrumentation designs, data types, and applications leads to unique challenges in obtaining well-controlled data sets for training and validation. Meanwhile, great efforts over the last four decades have focused on developing accurate and computationally efficient light propagation models that are flexible enough to simulate a wide variety of experimental conditions.
Aim: Recent developments in Monte Carlo (MC)-based modeling offer the unique advantage of simulating accurately light propagation spatially, temporally, and over an extensive range of optical parameters, including minimally to highly scattering tissue within a computationally efficient platform. Herein, we demonstrate how such MC platforms, namely “Monte Carlo eXtreme” and “Mesh-based Monte Carlo,” can be leveraged to generate large and representative data sets for training the DL model efficiently.
Approach: We propose data generator pipeline strategies using these platforms and demonstrate their potential in fluorescence optical topography, fluorescence optical tomography, and single-pixel diffuse optical tomography. These applications represent a large variety in instrumentation design, sample properties, and contrast function.
Results: DL models trained using the MC-based in silico datasets, validated further with experimental data not used during training, show accurate and promising results.
Conclusion: Overall, these MC-based data generation pipelines are expected to support the development of DL models for rapid, robust, and user-friendly image formation in a wide variety of applications.
KEYWORDS: Tumors, Tumor growth modeling, Multiplexing, Fluorescence resonance energy transfer, Systems modeling, Signal detection, Luminescence, Mode conditioning cables, In vivo imaging, Therapeutic antibodies
Macroscopic fluorescence lifetime FRET imaging (MFLI-FRET) presents a much-needed analytical tool to non-invasively quantify drug-receptor engagement in tumors and other organs in preclinical studies. We demonstrate the specificity and sensitivity of MFLI-FRET for direct and robust measurement of trastuzumab-HER2 target engagement in various types of breast and ovarian cancer tumor xenograft models. Simultaneous metabolic imaging with IRDye 800 CW 2-DG reveals that intracellular delivery of drug is associated with 2-DG lifetime and likely reflects tumors’ microenvironment and perfusion state.
KEYWORDS: Tumors, Fluorescence resonance energy transfer, Near infrared, Fluorescence lifetime imaging, Signal detection, Resonance energy transfer, Microscopy, Imaging systems
There is a continuing need to develop preclinical molecular imaging modalities to guide the development and optimization of targeted therapies in oncology. We have established Macroscopic Fluorescence Lifetime Imaging (MFLI) associated with Forster Resonance Energy Transfer (FRET) to report quantitatively on antibody-target engagement in live intact animals at the organ level. Here, we use FLI FRET imaging to quantify the binding of near infrared labeled antibody drugs to breast cancer cells or tumor xenografts. In combination with antibody engineering, this approach provides a robust analytical tool to optimize the binding and uptake of antibody-target binding during tumor progression and treatment.
KEYWORDS: In vivo imaging, Fluorescence lifetime imaging, Sensors, In vitro testing, Visible radiation, Tumors, Near infrared, Luminescence, Fluorescence resonance energy transfer, Detector development
Near-Infrared wide-field Fluorescence Lifetime Imaging (FLI) has become an increasingly popular method due to its unique specificity in sensing the cellular micro-environment and/or protein-protein interactions via FRET, but the approach is still challenging due to inefficient detection modules. Here, we report on the characterization of a large gated SPAD array, SwissSPAD2, towards in vivo preclinical imaging of FLI-FRET. Fluorescence decay fitting as well as phasor analysis are used to demonstrate the ability of SwissSPAD2 to accurately quantify short lifetimes and associated lifetime parameters in both in vitro and in vivo experiments, in full agreement with gated ICCD measurements.
In this study, we trained a convolutional neural network (CNN) utilizing a mix of recent CNN architectural design strategies. Our goals are to leverage these modern techniques to improve the binary classification of kidney tumor images obtained using Multi-Photon Microscopy (MPM). We demonstrate that incorporating these newer model design elements, coupled with transfer learning, image standardization, and data augmentation, leads to significantly increased classification performance over previous results. Our best model averages over 90% sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUROC) in image-level classification across cross-validation folds, superior to the previous best in all four metrics.
The performance of SwissSPAD2 (SS2), a large scale, widefield time-gated CMOS SPAD imager developed for fluorescence lifetime imaging, has recently been described in the context of visible range and fluorescence lifetime imaging microscopy (FLIM) of dyes with lifetimes in the 2.5 – 4 ns range. Here, we explore its capabilities in the NIR regime relevant for small animal imaging, where its sensitivity is lower and typical NIR fluorescent dye lifetimes are much shorter (1 ns or less). We carry out this study in a simple macroscopic imaging setup based on a compact NIR picosecond pulsed laser, an engineered diffuser-based illumination optics, and NIR optimized imaging lens suitable for well-plate or small animal imaging. Because laser repetition rates can vary between models, but the synchronization signal frequency accepted by SS2 is fixed to 20 MHz, we first checked that a simple frequency-division scheme enables data recording for different laser repetition rates. Next, we acquired data using different time gate widths, including gates with duration longer than the laser period, and analyzed the resulting data using both standard nonlinear least-square fit (NLSF) and phasor analysis. We show that the fixed synchronization rate and large gate widths characterizing SS2 (10 ns and over) are not an obstacle to accurately extracting lifetime in the 1 ns range and to distinguishing between close lifetimes. In summary, SS2 and similar very large gated SPAD imagers appear as a versatile alternative to other widefield time-resolved detectors for NIR fluorescence lifetime imaging, including preclinical molecular applications.
Significance: Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.
Aim: We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).
Approach: First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.
Results: The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.
Conclusions: The heavily validated capability of DL’s use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient’s bedside.
Overexpression of Human EGF Receptor 2 (HER2) in cancer is a marker of aggressive metastatic disease and poor prognosis. Anti-HER2 humanized monoclonal antibody trastuzumab (TZM) has been successfully used in the clinic over the last decades. However, a large fraction of eligible patients display resistance to this therapy. This calls for a deeper investigation of HER2 interaction with other members of HER tyrosine kinase receptors and modulation of their endocytic trafficking upon TZM treatment. Forster resonance energy transfer Fluorescence lifetime microscopy (FLIM- FRET) offers a robust approach to monitor HER2 homo and heterodimerization via the reduction of donor fluorophore lifetime. The objective of this study was to assess the dynamics of HER receptor homo and heterodimerization behavior via FLIMFRET by using near-infrared (NIR) FRET pair labeled anti-HER2 and anti-EGFR therapeutic antibodies in HER2- overexpressing breast cancer cells. In addition, we tested our new deep learning platform DL4FLIM adapted for automated analysis of all datasets. Herein, we report a first attempt to quantify NIR FRET pair labeled cetuximab (CTM, as a donor) and TZM (as an acceptor) binding to their receptors EGFR and HER2 respectively in AU565 cells. As a control, we also performed and human isotype IgG FLIM -FRET and found it completely non-specific. Our data demonstrate both the occurrence of FRET between NIR-labeled probes CTM and TZM as well as between CTM-CTM bound to their respective receptors. This proof-of -principal study demonstrated feasibility of monitoring HER2 hetero receptor FRET FLIM to better understand mechanism of TZM resistance.
Diffuse optical tomography, including fluorescence molecular tomography (FMT) have been greatly facilitated by the implementation of structured illumination (SI) strategies in recent years. In this work, we investigate the inverse problem in k-space reflectance fluorescence tomography. This in silico investigation leverages MCX, a Monte Carlo based platform, to generate large data sets for comparison between dAUTOMAP, a deep learning architecture, and commonly employed iterative solvers. We show that the proposed dAUTOMAP-based technique outperforms the traditional reconstruction algorithms. This new image formation approach is expected to facilitate imaging of sub-cutaneous tumors in live animals with enhanced resolution compared to the current gold standard.
KEYWORDS: Fluorescence lifetime imaging, Monte Carlo methods, Scattering, Process modeling, Near infrared, Luminescence, Absorption, Tissues, Photon transport
Herein, we report on a depth-resolved Macroscopic Fluorescence Lifetime Imaging (MFLI) analytic framework based around machine learning coupled with a computationally efficient Monte Carlo-based data simulation workflow for robust and user-friendly model training. Our Siamese convolutional neural network (CNN) takes both optical properties (OPs) and time-resolved fluorescence decays as input and reconstructs both lifetime maps and depth profiles simultaneously. We validate our approach using phantom embeddings in silico and experimentally using Spatial Frequency Domain Imaging (SFDI) for OP retrieval. To our knowledge, this is the first study reporting the augmentation of MFLI with wide-field SFDI for lifetime topography.
KEYWORDS: In vivo imaging, Auto-fluorescence imaging, Tissues, In vitro testing, Visible radiation, Structured light, Spectrophotometry, Single photon, Sensors, Near infrared
We propose a Single-Pixel Macroscopic Autofluorescence Imaging platform with supercontinuum excitation (440-690nm) and 16 parallel wavelength detection (475-1000nm) through a spectrophotometer coupled Single Photon Counting PMT. Recorded decays of FAD, POPOP and PPIX commercial auto-fluorophores serve to simulate training samples for UNMIX-ME, a deep learning algorithm that disentangles spectral overlaps. In silico mixed samples are reconstructed as a proof of concept and mixed in vitro samples prepared, measured and reconstructed to unmixed intensity and lifetime images. The results highlight the utility of the platform to macroscopically quantify autofluorescence lifetime in vitro and its future potential for in vivo autofluorescence imaging.
KEYWORDS: Luminescence, Fluorescence resonance energy transfer, Near infrared, Visible radiation, Resonance energy transfer, In vitro testing, In vivo imaging, 3D acquisition, 3D image processing, 3D metrology
Quantification of ligand-receptor engagement in human breast cancer cells and tumor xenografts has been performed using fluorescence lifetime Forster resonance energy transfer (FLI-FRET) imaging at multiscale, from in vitro microscopy to in vivo macroscopy and across visible to near-infrared wavelengths. We have developed a 3D convolutional neural network architecture, named FLI-Network (FLI-Net), to process fluorescence lifetime decays acquired by either Time-Correlated Single-Photon Counting (TCSCP)- or gated ICCD- based instruments. FLI-FRET ability to measure target engagement across different imaging platforms as well as post-processing analysis approaches can find numerous applications in pre-clinical drug delivery and targeted therapy assessment and optimization.
Human EGF receptor 2 (HER2) is an important oncogene and marker of aggressive metastatic cancer, found in up to 20% of oncologic patients. Anti-HER2 humanized monoclonal antibody trastuzumab (TZM) has been successfully used over the last two decades. However, both primary and acquired TZM resistance calls for the deeper investigation on TZMHER2 binding, internalization and trafficking/degradation in cancer cells in vitro and in vivo. Fluorescence lifetime FRET imaging (FLIM FRET) offers a unique approach to monitor TZM-HER2 binding followed by their uptake into target cells via the reduction of donor fluorophore lifetime. In this study, we characterized for the first time TZM-AF700 uptake and its relation to HER2 expression in AU565 human breast cancer cell line using confocal microscopy. Further, we have quantified the dimerization of HER2 via NIR FLIM FRET in vitro microscopy. Extensive analysis confirmed high specificity and efficiency of TZM FRET signal. Interestingly, we observed a significant heterogeneity of FRET within the cells: the highest TZM FRET levels occurred at the plasma membrane, whereas less if any donor lifetime reduction was registered in the perinuclear endosomes. These results suggest that HER2 dimers undergo dissociation or degradation upon TZM binding and trafficking. Overall, this study provides a good foundation for in vivo TZM FRET imaging of target engagement in preclinical studies.
Fluorescence lifetime imaging (FLI) has become an invaluable tool in the biomedical field by providing unique, quantitative information about biochemical events and interactions taking place within specimens of interest. Applications of FLI range from superresolution microscopy to whole body imaging using visible and near-infrared fluorophores. However, quantifying lifetime can still be a challenging task especially in the case of bi-exponential applications. In such cases, model based iterative fitting is typically employed but necessitate setting up multiple parameters ad hoc and can be computationally expensive. These facts have limited the universal appeal of the technique and methodologies can be specific to certain applications/technology or laboratory bound. Herein, we propose a novel approach based on Deep Learning (DL) to quantify bi-lifetime Forster Resonance Energy Transfer (FRET). Our deep neural network outputs three images consisting of both lifetimes and fractional amplitude. The network is trained using synthetic data and then validated on experimental FLI microscopic (FLIM) and macroscopic data sets (MFLI). Our results demonstrate that DL is well suited to quantify wide-field bi-exponential fluorescence lifetime accurately and in real time, even when it is difficult to obtain large scale experimental training data.
A clear distinction between oncocytoma and chromophobe renal cell carcinoma (chRCC) is critically important for clinical management of patients. But it may often be difficult to distinguish the two entities based on hematoxylin and eosin (H and E) stained sections alone. In this study, second harmonic generation (SHG) signals which are very specific to collagen were used to image collagen fibril structure. We conduct a pilot study to develop a new diagnostic method based on the analysis of collagen associated with kidney tumors using convolutional neural networks (CNNs). CNNs comprise a type of machine learning process well-suited for drawing information out of images. This study examines a CNN model’s ability to differentiate between oncocytoma (benign), and chRCC (malignant) kidney tumor images acquired with second harmonic generation (SHG), which is very specific for collagen matrix. To the best of our knowledge, this is the first study that attempts to distinguish the two entities based on their collagen structure. The model developed from this study demonstrated an overall classification accuracy of 68.7% with a specificity of 66.3% and sensitivity of 74.6%. While these results reflect an ability to classify the kidney tumors better than chance, further studies will be carried out to (a) better realize the tumor classification potential of this method with a larger sample size and (b) combining SHG with two-photon excited intrinsic fluorescence signal to achieve better classification.
Native fluorescence spectra play important roles in cancer detection. It is widely acknowledged that the emission spectrum of a tissue is a superposition of spectra of various salient fluorophores. However, component quantification is essentially an ill-posed problem. To address this problem, the native fluorescence spectra of normal human very low (LNCap), moderately metastatic (DU-145), and advanced metastatic (PC-3) cell lines were studied by the selected wavelength of 300 nm to investigate the key fluorescent molecules such as tryptophan, collagen and NADH. The native fluorescence spectra of cancer cell lines at different risk levels were analyzed using various machine learning algorithms for feature detection and develop criteria to separate the three types of cells. Principal component analysis (PCA), nonnegative matrix factorization (NMF), and partial least squares fitting were used separately to reduce dimension, extract features and detect biomolecular alterations reflected in the spectra. The scores corresponding to the basis spectra were used for classification. A linear support vector machine (SVM) was used to classify the spectra of the cells with different metastatic ability. In detection of signals coming from tryptophan and NADH with observed data corrupted by noise and inference, a sufficient statistic can be obtained based on the basis spectra retrieved using nonnegative matrix factorization. This work shows changes of relative contents of tryptophan and NADH obtained from native fluorescence spectroscopy may present potential criteria for detecting cancer cell lines of different metastatic ability.
Analyzing spectral or imaging data collected with various optical biopsy methods is often times difficult due to the complexity of the biological basis. Robust methods that can utilize the spectral or imaging data and detect the characteristic spectral or spatial signatures for different types of tissue is challenging but highly desired. In this study, we used various machine learning algorithms to analyze a spectral dataset acquired from human skin normal and cancerous tissue samples using resonance Raman spectroscopy with 532nm excitation. The algorithms including principal component analysis, nonnegative matrix factorization, and autoencoder artificial neural network are used to reduce dimension of the dataset and detect features. A support vector machine with a linear kernel is used to classify the normal tissue and cancerous tissue samples. The efficacies of the methods are compared.
Worldwide breast cancer incidence has increased by more than twenty percent in the past decade. It is also known that in that time, mortality due to the affliction has increased by fourteen percent. Using optical-based diagnostic techniques, such as Raman spectroscopy, has been explored in order to increase diagnostic accuracy in a more objective way along with significantly decreasing diagnostic wait-times. In this study, Raman spectroscopy with 532-nm excitation was used in order to incite resonance effects to enhance Stokes Raman scattering from unique biomolecular vibrational modes. Seventy-two Raman spectra (41 cancerous, 31 normal) were collected from nine breast tissue samples by performing a ten-spectra average using a 500-ms acquisition time at each acquisition location. The raw spectral data was subsequently prepared for analysis with background correction and normalization. The spectral data in the Raman Shift range of 750- 2000 cm-1 was used for analysis since the detector has highest sensitivity around in this range. The matrix decomposition technique nonnegative matrix factorization (NMF) was then performed on this processed data. The resulting leave-oneout cross-validation using two selective feature components resulted in sensitivity, specificity and accuracy of 92.6%, 100% and 96.0% respectively. The performance of NMF was also compared to that using principal component analysis (PCA), and NMF was shown be to be superior to PCA in this study. This study shows that coupling the resonance Raman spectroscopy technique with subsequent NMF decomposition method shows potential for high characterization accuracy in breast cancer detection.
Native fluorescence spectra are acquired from fresh normal and cancerous human prostate tissues. The fluorescence data
are analyzed using a multivariate analysis algorithm such as non-negative matrix factorization. The nonnegative spectral
components are retrieved and attributed to the native fluorophores such as collagen, reduced nicotinamide adenine
dinucleotide (NADH), and flavin adenine dinucleotide (FAD) in tissue. The retrieved weights of the components, e.g.
NADH and FAD are used to estimate the relative concentrations of the native fluorophores and the redox ratio. A
machine learning algorithm such as support vector machine (SVM) is used for classification to distinguish normal and
cancerous tissue samples based on either the relative concentrations of NADH and FAD or the redox ratio alone. The
classification performance is shown based on statistical measures such as sensitivity, specificity, and accuracy, along
with the area under receiver operating characteristic (ROC) curve. A cross validation method such as leave-one-out is
used to evaluate the predictive performance of the SVM classifier to avoid bias due to overfitting.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.