University of California

Yichen Wu

Aydogan Ozcan

Yibo Zhang

Yair Rivenson

Wei Luo

Steve Wei Feng

Harun Gunaydin

A trained deep neural network transforms an image of a sample obtained with a holographic microscope to an image that substantially resembles a microscopy image obtained with a microscope having a different microscopy image modality. Examples of different imaging modalities include bright-field, fluorescence, and dark-field. For bright-field applications, deep learning brings bright-field microscopy contrast to holographic images of a sample, bridging the volumetric imaging capability of holography with the speckle-free and artifact-free image contrast of bright-field microscopy. Holographic microscopy images obtained with a holographic microscope are input into a trained deep neural network to perform cross-modality image transformation from a digitally back-propagated hologram corresponding to a particular depth within a sample volume into an image that substantially resembles a microscopy image of the sample obtained at the same particular depth with a microscope having the different microscopy image modality.

System and method for transforming holographic microscopy images to microscopy images of various modalities

A fluorescence microscopy method includes a trained deep neural network. At least one 2D fluorescence microscopy image of a sample is input to the trained deep neural network, wherein the input image(s) is appended with a digital propagation matrix (DPM) that represents, pixel-by-pixel, an axial distance of a user-defined or automatically generated surface within the sample from a plane of the input image. The trained deep neural network outputs fluorescence output image(s) of the sample that is digitally propagated or refocused to the user-defined surface or automatically generated. The method and system cross-connects different imaging modalities, permitting 3D propagation of wide-field fluorescence image(s) to match confocal microscopy images at different sample planes. The method may be used to output a time sequence of images (e.g., time-lapse video) of a 2D or 3D surface within a sample.

Systems and methods for two-dimensional fluorescence wave propagation onto surfaces using deep learning

A method of performing phase retrieval and holographic image reconstruction of an imaged sample includes obtaining a single hologram intensity image of the sample using an imaging device. The single hologram intensity image is back-propagated to generate a real input image and an imaginary input image of the sample with image processing software, wherein the real input image and the imaginary input image contain twin-image and/or interference-related artifacts. A trained deep neural network is provided that is executed by the image processing software using one or more processors and configured to receive the real input image and the imaginary input image of the sample and generate an output real image and an output imaginary image in which the twin-image and/or interference-related artifacts are substantially suppressed or eliminated. In some embodiments, the trained deep neural network simultaneously achieves phase-recovery and auto-focusing significantly extending the DOF of holographic image reconstruction.

Method and system for phase recovery and holographic image reconstruction using a neural network

A method of generating a color image of a sample includes obtaining a plurality of low resolution holographic images of the sample using a color image sensor, the sample illuminated simultaneously by light from three or more distinct colors, wherein the illuminated sample casts sample holograms on the image sensor and wherein the plurality of low resolution holographic images are obtained by relative x, y, and z directional shifts between sample holograms and the image sensor. Pixel super-resolved holograms of the sample are generated at each of the three or more distinct colors. De-multiplexed holograms are generated from the pixel super-resolved holograms. Phase information is retrieved from the de-multiplexed holograms using a phase retrieval algorithm to obtain complex holograms. The complex hologram for the three or more distinct colors is digitally combined and back-propagated to a sample plane to generate the color image.

Method and system for pixel super-resolution of multiplexed holographic color images

A label-free bio-aerosol sensing platform and method uses a field-portable and cost-effective device based on holographic microscopy and deep-learning, which screens bio-aerosols at a high throughput level. Two different deep neural networks are utilized to rapidly reconstruct the amplitude and phase images of the captured bio-aerosols, and to output particle information for each bio-aerosol that is imaged. This includes, a classification of the type or species of the particle, particle size, particle shape, particle thickness, or spatial feature(s) of the particle. The platform was validated using the label-free sensing of common bio-aerosol types, e.g., Bermuda grass pollen, oak tree pollen, ragweed pollen,spore, andspore and achieved &#x3e;94% classification accuracy. The label-free bio-aerosol platform, with its mobility and cost-effectiveness, will find several applications in indoor and outdoor air quality monitoring.

Label-free bio-aerosol sensing using mobile microscopy and deep learning

A lens-free microscope for monitoring air quality includes a housing that contains a vacuum pump configured to draw air into an impaction nozzle. The impaction nozzle has an output located adjacent to an optically transparent substrate for collecting particles. One or more illumination sources are disposed in the housing and are configured to illuminate the collected particles on the optically transparent substrate. An image sensor is located adjacent to the optically transparent substrate, wherein the image sensor collects particle diffraction patterns or holographic images cast upon the image sensor. At least one processor is disposed in the housing and controls the vacuum pump and the one or more illumination sources. Image files are transferred to a separate computing device for image processing using machine learning to identify particles and perform data analysis to output particle images, particle size, particle density, and/or particle type data.

Mobile microscopy system for air quality monitoring

Methods and systems for generating a high-color-fidelity and high-resolution color image of a sample are disclosed; which fuses or merges a holographic image acquired at a single wavelength with a color-calibrated image taken by a low-magnification lens-based microscope using a wavelet transform based colorization method. A holographic microscope is used to obtain holographic images which are used to computationally reconstruct a high-resolution mono-color holographic image of the sample. A lens-based microscope is used to obtain low resolution color images. A discrete wavelet transform (DWT) is used to generate a final image that merges the low-resolution components from the lens-based color image and the high-resolution components from the high-resolution mono-color holographic image.

Growing community of inventors

Los Angeles, CA, United States of America

Yichen Wu