Leica Biosystems Imaging, Inc.

Walter Georgescu

Allen Olson

Kiran Saligrama

Bharat Annaldas

Kevin Shields

Mark Gregson

Darragh Lawler

Girish Mallya Udupi

Bruno Oliveira

A computer apparatus and method for identifying and visualizing tumors in a histological image and measuring a tumor margin are provided. A CNN is used to classify pixels in the image according to whether they are determined to relate to nontumorous tissue, or one or more classes for tumorous tissue. Segmentation is carried out based on the CNN results to generate a mask that marks areas occupied by individual tumors. Summary statistics for each tumor are computed and supplied to a filter which edits the segmentation mask by filtering out tumors deemed to be insignificant. Optionally, the tumors that pass the filter may be ranked according to the summary statistics, for example in order of clinical relevance or by a sensible order of review for a pathologist. A visualization application can then display the histological image having regard to the segmentation mask, summary statistics and/or ranking. Tumor masses extracted by resection are painted with an ink to highlight its surface region. The CNN is trained to distinguish ink and no-ink tissue as well as tumor and no-tumor tissue. The CNN is applied to the histological image to generate an output image whose pixels are assigned to the tissue classes. Tumor margin status of the tissue section is determined by the presence or absence of tumor-and-ink classified pixels. Tumor margin involvement and tumor margin distance are determined by computing additional parameters based on classification-specified inter-pixel distance parameters.

Computer supported review of tumors in histology images and post operative tumor margin assessment

A CNN is applied to a histological image to identify areas of interest. The CNN classifies pixels according to relevance classes including one or more classes indicating levels of interest and at least one class indicating lack of interest. The CNN is trained on a training data set including data which has recorded how pathologists have interacted with visualizations of histological images. In the trained CNN, the in-terest-based pixel classification is used to generate a segmentation mask that defines areas of interest. The mask can be used to indicate where in an image clinically relevant features may be located. Further, it can be used to guide variable data compression of the histological image. Moreover, it can be used to control loading of image data in either a client-server model or within a memory cache policy. Furthermore, a histological image of a tissue sample of a tissue type that has been treated with a test compound is image processed in order to detect areas where toxic reactions to the test compound may have occurred. An autoencoder is trained with a training data set comprising histological images of tissue samples which are of the given tissue type, but which have not been treated with the test compound. The trained autoencoder is applied to detect tissue areas by their deviation from the normal variation seen in that tissue type as learnt by the training process, and so build up a toxicity map of the image. The toxicity map can then be used to direct a toxicological pathologist to examine the areas identified by the autoencoder as lying outside the normal range of heterogeneity for the tissue type. This makes the pathologist's review quicker and more reliable. The toxicity map can also be overlayed with the segmentation mask indicating areas of interest. When an area of interest and an area identified as lying outside the normal range of heterogeneity for the tissue type, and increased confidence score is applied to the overlapping area.

Neural network based identification of areas of interest in digital pathology images

A computer apparatus and method for identifying and visualizing tumors in a histological image and measuring a tumor margin are provided. A CNN is used to classify pixels in the image according to whether they are determined to relate to non-tumorous tissue, or one or more classes for tumorous tissue. Segmentation is carried out based on the CNN results to generate a mask that marks areas occupied by individual tumors. Summary statistics for each tumor are computed and supplied to a filter which edits the segmentation mask by filtering out tumors deemed to be insignificant. Optionally, the tumors that pass the filter may be ranked according to the summary statistics, for example in order of clinical relevance or by a sensible order of review for a pathologist. A visualization application can then display the histological image having regard to the segmentation mask, summary statistics and/or ranking. Tumor masses extracted by resection are painted with an ink to highlight its surface region. The CNN is trained to distinguish ink and no-ink tissue as well as tumor and no-tumor tissue. The CNN is applied to the histological image to generate an output image whose pixels are assigned to the tissue classes. Tumor margin status of the tissue section is determined by the presence or absence of tumor-and-ink classified pixels. Tumor margin involvement and tumor margin distance are determined by computing additional parameters based on classification-specified inter-pixel distance parameters.

A convolutional neural network (CNN) is applied to identifying tumors in a histological image. The CNN has one channel assigned to each of a plurality of tissue classes that are to be identified, there being at least one class for each of non-tumorous and tumorous tissue types. Multi-stage convolution is performed on image patches extracted from the histological image followed by multi-stage transpose convolution to recover a layer matched in size to the input image patch. The output image patch thus has a one-to-one pixel-to-pixel correspondence with the input image patch such that each pixel in the output image patch has assigned to it one of the multiple available classes. The output image patches are then assembled into a probability map that can be co-rendered with the histological image either alongside it or over it as an overlay. The probability map can then be stored linked to the histological image.

Processing of histology images with a convolutional neural network to identify tumors

Automated stain finding. In an embodiment, an image of a sample comprising one or more stains is received. For each of a plurality of pixels in the image, an optical density vector for the pixel is determined. The optical density vector comprises a value for each of the one or more stains, and represents a point in an optical density space that has a number of dimensions equal to a number of the one or more stains. The optical density vectors are transformed from the optical density space into a representation in a lower dimensional space. The lower dimensional space has a number of dimensions equal to one less than the number of dimensions of the optical density space. An optical density vector corresponding to each of the one or more stains is identified based on the representation.

Automated stain finding in pathology bright-field images

Automatic nuclear segmentation. In an embodiment, a plurality of superpixels are determined in a digital image. For each of the superpixels, any superpixels located within a search radius from the superpixel are identified, and, for each unique local combination between the superpixel and any identified superpixels located within the search radius from the superpixel, a local score for the local combination is determined. One of a plurality of global sets of local combinations with an optimum global score is identified based on the determined local scores.

Growing community of inventors

San Marcos, CA, United States of America

Walter Georgescu