Arizona State University

Mayo Foundation for Medical Education and Research

Arizona Board of Regents, a Body Corporate

Nima Tajbakhsh

Jianming Liang

Suryakanth R Gurudu

Zongwei Zhou

Md Mahfuzur Rahman Siddiquee

Hong Wu

Wenzhe Xue

Ruibin Feng

Marek Belohlavek

Jae Shin

Eileen M McMahon

Vatsal Sodha

Changching Chi

Haripriya Sharma

Jaeyul Shin

Described herein are means for generating source models for transfer learning to application specific models used in the processing of medical imaging. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample in the group of training samples includes an image; for each training sample in the group of training samples: identifying an original patch of the image corresponding to the training sample; identifying one or more transformations to be applied to the original patch; generating a transformed patch by applying the one or more transformations to the identified patch; and training an encoder-decoder network using a group of transformed patches corresponding to the group of training samples, wherein the encoder-decoder network is trained to generate an approximation of the original patch from a corresponding transformed patch, and wherein the encoder-decoder network is trained to minimize a loss function that indicates a difference between the generated approximation of the original patch and the original patch. The source models significantly enhance the transfer learning performance for many medical imaging tasks including, but not limited to, disease/organ detection, classification, and segmentation. Other related embodiments are disclosed.

Systems, methods, and apparatuses for the generation of source models for transfer learning to application specific models used in the processing of medical imaging

Described herein are means for implementing fixed-point image-to-image translation using improved Generative Adversarial Networks (GANs). For instance, an exemplary system is specially configured for implementing a new framework, called a Fixed-Point GAN, which improves upon prior known methodologies by enhancing the quality of the images generated through global, local, and identity transformation. The Fixed-Point GAN as introduced and described herein, improves many applications dependant on image-to-image translation, including those in the field of medical image processing for the purposes of disease detection and localization. Other related embodiments are disclosed.

Systems, methods, and apparatuses for implementing fixed-point image-to-image translation using improved generative adversarial networks (GANs)

Methods, systems, and media for segmenting images are provided. In some embodiments, the method comprises: generating an aggregate U-Net comprised of a plurality of U-Nets, wherein each U-Net in the plurality of U-Nets has a different depth, wherein each U-Net is comprised of a plurality of nodes X, wherein i indicates a down-sampling layer the U-Net, and wherein j indicates a convolution layer of the U-Net; training the aggregate U-Net by: for each training sample in a group of training samples, calculating, for each node in the plurality of nodes X, a feature map x, wherein xis based on a convolution operation performed on a down-sampling of an output from Xwhen j=0, and wherein xis based on a convolution operation performed on an up-sampling operation of an output from Xwhen j&#x3e;0; and predicting a segmentation of a test image using the trained aggregate U-Net.

Methods, systems, and media for segmenting images

Methods, systems, and media for discriminating and generating translated images are provided. In some embodiments, the method comprises: identifying a set of training images, wherein each image is associated with at least one domain from a plurality of domains; training a generator network to generate: i) a first fake image that is associated with a first domain; and ii) a second fake image that is associated with a second domain; training a discriminator network, using as inputs to the discriminator network: i) an image from the set of training images; ii) the first fake image; and iii) the second fake image; and using the generator network to generate, for an image not included in the set of training images at least one of: i) a third fake image that is associated with the first domain; and ii) a fourth fake image that is associated with the second domain.

Methods, systems, and media for discriminating and generating translated images

Disclosed are provided systems, methods, and apparatuses for implementing a multi-resolution neural network for use with imaging intensive applications including medical imaging. For example, a system having means to execute a neural network model formed from a plurality of layer blocks including an encoder layer block which precedes a plurality of decoder layer blocks includes: means for associating a resolution value with each of the plurality of layer blocks; means for processing via the encoder layer block a respective layer block input including a down-sampled layer block output processing, via decoder layer blocks, a respective layer block input including an up-sampled layer block output and a layer block output of a previous layer block associated with a prior resolution value of a layer block which precedes the respective decoder layer block; and generating the respective layer block output by summing or concatenating the processed layer block inputs.

Systems, methods, and apparatuses for implementing a multi-resolution neural network for use with imaging intensive applications including medical imaging

Detecting a pulmonary embolism (PE) in an image dataset of a blood vessel involves obtaining a volume of interest (VOI) in the blood vessel, generating a plurality of PE candidates within the VOI, generating a set of voxels for each PE candidate, estimating for each PE candidate an orientation of the blood vessel that contains the PE candidate, given the set of voxels for the PE candidate, and generating a visualization of the blood vessel that contains the PE candidate using the estimated orientation of the blood vessel that contains the PE candidate.

Method and apparatus for detection and visualization of pulmonary embolism

Mechanisms for simultaneously monitoring colonoscopic video quality and detecting polyps in colonoscopy are provided. In some embodiments, the mechanisms can include a quality monitoring system that uses a first trained classifier to monitor image frames from a colonoscopic video to determine which image frames are informative frames and which image frames are non-informative frames. The informative image frames can be passed to an automatic polyp detection system that uses a second trained classifier to localize and identify whether a polyp or any other suitable object is present in one or more of the informative image frames.

Methods, systems, and media for simultaneously monitoring colonoscopic video quality and detecting polyps in colonoscopy

A system and methods for polyp detection using optical colonoscopy images are provided. In some aspects, the system includes an input configured to receive a series of optical images, and a processor configured to process the series of optical images with steps comprising of receiving an optical image from the input, constructing an edge map corresponding to the optical image, the edge map comprising a plurality of edge pixel, and generating a refined edge map by applying a classification scheme based on patterns of intensity variation to the plurality of edge pixels in the edge map. The processor may also process the series with steps of identifying polyp candidates using the refined edge map, computing probabilities that identified polyp candidates are polyps, and generating a report, using the computed probabilities, indicating detected polyps. The system also includes an output for displaying the report.

System and method for automatic polyp detection using global geometric constraints and local intensity variation patterns

A system and methods for detecting polyps using optical images acquired during a colonoscopy. In some aspects, a method includes receiving the set of optical images from the input and generating polyp candidates by analyzing the received set of optical images. The method also includes generating a plurality of image patches around locations associated with each polyp candidate, applying a set of convolutional neural networks to the corresponding image patches, and computing probabilities indicative of a maximum response for each convolutional neural network. The method further includes identifying polyps using the computed probabilities for each polyp candidate, and generating a report indicating identified polyps.

System and methods for automatic polyp detection using convulutional neural networks

A system for quality assessment of optical colonoscopy images includes an input device configured to acquire a series of images during an optical colonoscopy. A computing device is coupled in communication with the input device and configured to acquire from the input device an input image from the series of images captured during the optical colonoscopy; form a cell grid including a plurality of cells on the input image; perform an image transformation onto the input image with each cell of the plurality of cells within the cell grid; reconstruct each cell to form a reconstructed image; compute a difference image of a sum of a plurality of differences between the input image and the reconstructed image; compute a histogram of the difference image; and apply a probabilistic classifier to the histogram to calculate an informativeness score for the input image.

System and method for quality assessment of optical colonoscopy images

A system and method for detecting pulmonary embolisms in a subject's vasculature are provided. In some aspects, the method includes acquiring a set of images representing a vasculature of the subject, and analyzing the set of images to identify pulmonary embolism candidates associated with the vasculature. The method also includes generating, for identified pulmonary embolism candidates, image patches based on a vessel-aligned image representation, and applying a set of convolutional neural networks to the generated image patches to identify pulmonary embolisms. The method further includes generating a report indicating identified pulmonary embolisms.

System and method for automatic pulmonary embolism detection

A system for automatically determining a thickness of a wall of an artery of a subject includes an ECG monitoring device that captures an electrocardiogram (ECG) signal from the subject, and an ultrasound video imaging device, coupled to the ECG monitoring device, that receives the ECG signal from the ECG monitoring device, and captures a corresponding ultrasound video of the wall of the artery of the subject. The system produces a plurality of frames of video comprising the ultrasound video of the wall of the artery of the subject and an image of the ECG signal. A processor is configured to select a subset of the plurality of frames of the ultrasound video based on the image of the (ECG) signal, locate automatically a region of interest (ROI) in each frame of the subset of the plurality of frames of the video using a machine-based artificial neural network and measure automatically a thickness of the wall of the artery in each ROI using the machine-based artificial neural network.

Method and apparatus for video interpretation of carotid intima-media thickness

A system and method for automated polyp detection in optical colonoscopy images is provided. In one embodiment, the system and method for polyp detection is based on an observation that image appearance around polyp boundaries differs from that of other boundaries in colonoscopy images. To reduce vulnerability against misleading objects, the image processing method localizes polyps by detecting polyp boundaries, while filtering out irrelevant boundaries, with a generative-discriminative model. To filter out irrelevant boundaries, a boundary removal mechanism is provided that captures changes in image appearance across polyp boundaries. Thus, in this embodiment the boundary removal mechanism is minimally affected by texture visibility limitations. In addition, a vote accumulation scheme is applied that enables polyp localization from fragmented edge segmentation maps without identification of whole polyp boundaries.

System and method for detecting polyps from learned boundaries

A system and method is provided for automated polyp detection in optical colonoscopy images. The system includes an input configured to acquire a series of optical images, and a processor configured to process the optical images. Processing steps include performing a boundary classification with steps comprising locating a series of edge pixels using at least one acquired optical image, selecting an image patch around each said edge pixel, performing a classification threshold analysis on each image patch of said edge pixels using a set of determined boundary classifiers, and identifying, based on the classification threshold analysis, polyp edge pixels consistent with a polyp edge. Processing steps for the processor also include performing a vote accumulation, using the identified polyp edge pixels, to determine a polyp location. The system also includes an output configured to indicate potential polyps using the determined polyp location.

Growing community of inventors

Tempe, AZ, United States of America

Nima Tajbakhsh