Paige.ai, Inc.

Memorial Sloan Kettering Cancer Center

Donghia Furniture/textiles Ltd.

Thomas Fuchs

Leo John Grady

Christopher Kanan

Patricia Raciti

Sherri Donghia

Jennifer J Hutton

Belma Dogdas

Jillian Sue

Masaru Suzuki

Gabriele Campanella

Paul Mathieu

Supriya Kapur

Rodrigo Ceballos Lentini

Ran Godrich

Peter Schueffler

David Joon Ho

Peter J Schüffler

Brandon Rothrock

Andrew Schaumberg

Julian Viret

Razik Yousfi

Alican Bozkurt

Dig Vijay Kumar Yarlagadda

Matthew Lee

Chad Vanderbilt

Jorge Sergio Reis-Filho

Joe D'Urso

Matthew Hanna

Jason Locke

Jorge S Reis-filho

Hassan Muhammad

Kenan Turnacioglu

Chao Feng

Charles Ross Schmidtlein

Ida Häggström

Leo Grady

The present disclosure is directed to systems and methods for identifying regions of interest (ROIs) in images. A computing system may identify an image including an annotation defining an ROI. The image may have a plurality of pixels in a first color space. The computing system may convert the plurality of pixels from the first color space to a second color space to differentiate the annotation from the ROI. The computing system may select a first subset of pixels corresponding to the annotation based at least on a color value of the first subset of pixels in the second color space. The computing system may identify a second subset of pixels included in the ROI from the image using the first subset of pixels. The computing system may store an association between the second subset of pixels and the ROI defined by the annotation in the image.

Detection of annotated regions of interest in images

The present disclosure relates generally to image viewers, in particular biomedical images viewers that can concurrently render biomedical images at various magnifications or resolutions. The computing system can identify tiles from a first portion of a biomedical image. Each tile can correspond to a magnification level and coordinates in the biomedical image. The computing system can provide the tiles for concurrent display in respective graphical user interface (GUI) elements. The computing system can detect an interaction with a GUI element. The computing system can identify a change in coordinates for a tile displayed in the GUI element based on the interaction. The computing system can determine a second change in the other concurrently displayed tiles based on the change in coordinates. The computing system can update the concurrent display of the tiles based on the changes.

Universal slide viewer

Presented herein are systems and methods for classifying features from biomedical images. A computing system may identify a first portion corresponding to an ROI in a first biomedical image derived from a sample. The ROI of the first biomedical image may correspond to a feature of the sample. The computing system may generate a first embedding vector using the first portion of the first biomedical image. The computing system may apply the first embedding vector to a clustering model. The clustering model may have a feature space to define a plurality of conditions. The clustering model may be trained using a second embedding vectors generated from a corresponding second portions with at least one of a plurality of image transformation. The computing system may determine a condition for the feature based on applying the first embedding vector to the clustering model.

Learning representations of nuclei in histopathology images with contrastive loss

Systems and methods are disclosed for receiving a target electronic image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient, applying a machine learning system to the target electronic image to identify a region of interest of the target specimen and determine an expression level of, category of, and/or presence of a biomarker in the region of interest, the biomarker comprising at least one from among an epithelial growth factor receptor (EGFR) biomarker and/or a DNA mismatch repair (MMR) deficiency biomarker, the machine learning system having been generated by processing a plurality of training images to predict whether a region of interest is present in the target electronic image, the training images comprising images of human tissue and/or images that are algorithmically generated, and outputting the determined expression level of, category of, and/or presence of the biomarker in the region of interest.

Systems and methods for processing electronic images to infer biomarkers

The present application relates generally to identifying regions of interest in images, including but not limited to whole slide image region of interest identification, prioritization, de-duplication, and normalization via interpretable rules, nuclear region counting, point set registration, and histogram specification color normalization. This disclosure describes systems and methods for analyzing and extracting regions of interest from images, for example biomedical images depicting a tissue sample from biopsy or ectomy. Techniques directed to quality control estimation, granular classification, and coarse classification of regions of biomedical images are described herein. Using the described techniques, patches of images corresponding to regions of interest can be extracted and analyzed individually or in parallel to determine pixels correspond to features of interest and pixels that do not. Patches that do not include features of interest, or include disqualifying features, can be disqualified from further analysis. Relevant patches can analyzed and stored with various feature parameters.

Identifying regions of interest from whole slide images

The present disclosure is directed to systems and methods for classifying biomedical images. A feature classifier may generate a plurality of tiles from a biomedical image. Each tile may correspond to a portion of the biomedical image. The feature classifier may select a subset of tiles from the plurality of tiles by applying an inference model. The subset of tiles may have highest scores. Each score may indicate a likelihood that the corresponding tile includes a feature indicative of the presence of the condition. The feature classifier may determine a classification result for the biomedical image by applying an aggregation model. The classification result may indicate whether the biomedical includes the presence or lack of the condition.

Systems and methods for multiple instance learning for classification and localization in biomedical imaging

Systems and methods are disclosed for processing digital images to predict at least one continuous value comprising receiving one or more digital medical images, determining whether the one or more digital medical images includes at least one salient region, upon determining that the one or more digital medical images includes the at least one salient region, predicting, by a trained machine learning system, at least one continuous value corresponding to the at least one salient region, and outputting the at least one continuous value to an electronic storage device and/or display.

Systems and methods to process electronic images for continuous biomarker prediction

Systems and methods are disclosed for receiving digital images of a pathology specimen from a patient, the pathology specimen comprising tumor tissue, the one or more digital images being associated with data about a plurality of biomarkers in the tumor tissue and data about a surrounding invasive margin around the tumor tissue; identifying the tumor tissue and the surrounding invasive margin region to be analyzed for each of the one or more digital images; generating, using a machine learning model on the one or more digital images, at least one inference of a presence of the plurality of biomarkers in the tumor tissue and the surrounding invasive margin region; determining a spatial relationship of each of the plurality of biomarkers identified in the tumor tissue and the surrounding invasive margin region to themselves and to other cell types; and determining a prediction for a treatment outcome and/or at least one treatment recommendation for the patient.

Systems and methods for processing electronic images for biomarker localization

Systems and methods for processing images of slides to infer biomarkers

Described herein are Deep Multi-Magnification Networks (DMMNs). The multi-class tissue segmentation architecture processes a set of patches from multiple magnifications to make more accurate predictions. For the supervised training, partial annotations may be used to reduce the burden of annotators. The segmentation architecture with multi-encoder, multi-decoder, and multi-concatenation outperforms other segmentation architectures on breast datasets, and can be used to facilitate pathologists' assessments of breast cancer in margin specimens.

Deep multi-magnification networks for multi-class image segmentation

Systems and methods are disclosed for receiving a target image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient, applying a machine learning model to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the machine learning model having been generated by processing a plurality of training images to predict at least one characteristic, the training images comprising images of human tissue and/or images that are algorithmically generated, and outputting the at least one characteristic of the target specimen and/or the at least one characteristic of the target image.

Systems and methods for processing images to classify the processed images for digital pathology

Systems and methods are disclosed for generating a specialized machine learning model by receiving a generalized machine learning model generated by processing a plurality of first training images to predict at least one cancer characteristic, receiving a plurality of second training images, the first training images and the second training images include images of tissue specimens and/or images algorithmically generated to replicate tissue specimens, receiving a plurality of target specialized attributes related to a respective second training image of the plurality of second training images, generating a specialized machine learning model by modifying the generalized machine learning model based on the plurality of second training images and the target specialized attributes, receiving a target image corresponding to a target specimen, applying the specialized machine learning model to the target image to determine at least one characteristic of the target image, and outputting the characteristic of the target image.

Systems and methods for processing electronic images for generalized disease detection

Systems and methods are disclosed for receiving one or more digital images associated with a tissue specimen, detecting one or more image regions from a background of the one or more digital images, determining a prediction, using a machine learning system, of whether at least one first image region of the one or more image regions comprises at least one external contaminant, the machine learning system having been trained using a plurality of training images to predict a presence of external contaminants and/or a location of any external contaminants present in the tissue specimen, and determining, based on the prediction of whether a first image region comprises an external contaminant, whether to process the image region using an processing algorithm.

Systems and methods for processing electronic images to detect contamination in specimen preparations

The present application relates generally to image tiling, including but not limited to systems and methods of fast whole slide tissue tiling. A computing system may identify a first image of a first dimension from which to select one or more tiles. The computing system may perform a reduction operation on the first image to generate a second image of a second dimension. The computing system may apply a thresholding operation on the second image to identify a first set of pixels corresponding to the presence of the feature and a second set of pixels corresponding to the absence of the feature based on an intensity of each pixel in the second image. The computing system may select, from a plurality of tiles corresponding to the first image, a subset of tiles corresponding to the first set of pixels identified from the second image.

Fast whole slide tissue tiling method

The present disclosure is directed to systems and methods for reconstructing biomedical images. A projection preparer may identify a projection dataset derived from a tomographic biomedical imaging scan. An encoder-decoder model may reconstruct reconstructing tomographic biomedical images from projection data. The encoder-decoder model may include an encoder. The encoder may include a first series of transform layers to generate first feature maps using the projection dataset. The encoder-decoder may include a decoder. The decoder may include a second series of transform layers to generate second features maps using the first feature maps from the encoder. The system may include a reconstruction engine executable on the one or more processors. The reconstruction engine may apply the encoder-decoder model to the projection dataset to generate a reconstructed tomographic biomedical image based on the second feature maps generated by the decoder of the encoder-decoder model.

Growing community of inventors

New York, NY, United States of America

Thomas Fuchs