Leica Biosystems Imaging, Inc.

Aperio Technologies, Incorporated

Kiran Saligrama

Allen Olson

Yunlu Zou

Peyman Najmabadi

Walter Georgescu

Dirk G Soenksen

Bharat Annaldas

Greg J Crandall

Ruby Chen

Kevin Shields

Mark Gregson

Darragh Lawler

Girish Mallya Udupi

Bruno Oliveira

Greg Crandall

A CNN is applied to a histological image to identify areas of interest. The CNN classifies pixels according to relevance classes including one or more classes indicating levels of interest and at least one class indicating lack of interest. The CNN is trained on a training data set including data which has recorded how pathologists have interacted with visualizations of histological images. In the trained CNN, the in-terest-based pixel classification is used to generate a segmentation mask that defines areas of interest. The mask can be used to indicate where in an image clinically relevant features may be located. Further, it can be used to guide variable data compression of the histological image. Moreover, it can be used to control loading of image data in either a client-server model or within a memory cache policy. Furthermore, a histological image of a tissue sample of a tissue type that has been treated with a test compound is image processed in order to detect areas where toxic reactions to the test compound may have occurred. An autoencoder is trained with a training data set comprising histological images of tissue samples which are of the given tissue type, but which have not been treated with the test compound. The trained autoencoder is applied to detect tissue areas by their deviation from the normal variation seen in that tissue type as learnt by the training process, and so build up a toxicity map of the image. The toxicity map can then be used to direct a toxicological pathologist to examine the areas identified by the autoencoder as lying outside the normal range of heterogeneity for the tissue type. This makes the pathologist's review quicker and more reliable. The toxicity map can also be overlayed with the segmentation mask indicating areas of interest. When an area of interest and an area identified as lying outside the normal range of heterogeneity for the tissue type, and increased confidence score is applied to the overlapping area.

Neural network based identification of areas of interest in digital pathology images

Image viewing in digital pathology using eye-tracking. In an embodiment, a position of a user's gaze on a graphical user interface, comprising at least a portion of a digital slide image within a macro view, is repeatedly detected based on an output from an eye-tracking device. After detecting a change of the user's gaze from a first position to a second position on the graphical user interface, a view of the digital slide image within the macro view is automatically panned based on the second position, so as to move a position on the digital slide image that corresponds to the second position on the graphical user interface toward a center of the macro view.

Eye-tracking image viewer for digital pathology

A digital scanning apparatus is provided that includes imaging and focusing sensors and a processor to analyze the image data captured by the imaging and focusing sensors and adjust the focus of the scanning apparatus in real time during a scanning operation. The individual pixels of the imaging sensor are all in the same image plane with respect to the optical path of the digital scanning apparatus. The individual pixels of the focusing sensor are each in a different image plane with respect to the optical path, and one pixel of the focusing sensor is on the same image plane as the image sensor. The processor analyzes image data from the imaging sensor and the focusing sensor and determines a distance and direction to adjust the relative position of an objective lens and a stage of the digital scanning apparatus to achieve optimal focus during the scanning operation.

Real-time autofocus focusing algorithm

A convolutional neural network (CNN) is applied to identifying tumors in a histological image. The CNN has one channel assigned to each of a plurality of tissue classes that are to be identified, there being at least one class for each of non-tumorous and tumorous tissue types. Multi-stage convolution is performed on image patches extracted from the histological image followed by multi-stage transpose convolution to recover a layer matched in size to the input image patch. The output image patch thus has a one-to-one pixel-to-pixel correspondence with the input image patch such that each pixel in the output image patch has assigned to it one of the multiple available classes. The output image patches are then assembled into a probability map that can be co-rendered with the histological image either alongside it or over it as an overlay. The probability map can then be stored linked to the histological image.

Processing of histology images with a convolutional neural network to identify tumors

System for acquiring a digital image of a sample on a microscope slide. In an embodiment, the system comprises a stage configured to support a sample, an objective lens having a single optical axis that is orthogonal to the stage, an imaging sensor, and a focusing sensor. The system further comprises at least one beam splitter optically coupled to the objective lens and configured to receive a field of view corresponding to the optical axis of the objective lens, and simultaneously provide at least a first portion of the field of view to the imaging sensor and at least a second portion of the field of view to the focusing sensor. The focusing sensor may simultaneously acquire image(s) at a plurality of different focal distances and/or simultaneously acquire a pair of mirrored images, each comprising pixels acquired at a plurality of different focal distances.

Real-time focusing in line scan imaging

Automated stain finding. In an embodiment, an image of a sample comprising one or more stains is received. For each of a plurality of pixels in the image, an optical density vector for the pixel is determined. The optical density vector comprises a value for each of the one or more stains, and represents a point in an optical density space that has a number of dimensions equal to a number of the one or more stains. The optical density vectors are transformed from the optical density space into a representation in a lower dimensional space. The lower dimensional space has a number of dimensions equal to one less than the number of dimensions of the optical density space. An optical density vector corresponding to each of the one or more stains is identified based on the representation.

Automated stain finding in pathology bright-field images

Automatic nuclear segmentation. In an embodiment, a plurality of superpixels are determined in a digital image. For each of the superpixels, any superpixels located within a search radius from the superpixel are identified, and, for each unique local combination between the superpixel and any identified superpixels located within the search radius from the superpixel, a local score for the local combination is determined. One of a plurality of global sets of local combinations with an optimum global score is identified based on the determined local scores.

Automatic nuclear segmentation

Systems, methods, and media for assessing the quality of a microscope slide image. In an embodiment, a plurality of focus point values are acquired for a sample on a microscope slide. Each focus point value comprises x-y coordinates indicating a location on the sample and a z coordinate indicating a focus height for the location on the sample. A best-fit surface is calculated based on the focus point values, and it is determined whether or not outlying focus point values exist based on the best-fit surface and the z coordinate for one or more of the focus point values. A scanned image of the microscope slide may be displayed which comprises, for each outlying focus point value, an overlay that identifies the location on the sample of the outlying focus point value based on the x-y coordinates for the outlying focus point value.

Optimizing virtual slide image quality

Systems and methods for assessing and optimizing virtual microscope slide image quality are provided. In order to determine whether any of multiple virtual slide images has an out of focus area and is therefore a candidate for manual inspection or rescanning, various focus points used to scan each virtual slide image are used to calculate a best fit surface for each virtual slide image. The best fit surface is then used to determine whether any of the various focus points are outliers. If it is determined that a virtual slide image is associated with outlying focus points, it is identified as a candidate for manual inspection or rescanning.

Systems and methods for assessing and optimizing virtual microscope slide image quality are provided. In order to determine whether a virtual slide image has any out of focus areas and is therefore a candidate for manual inspection, the various focus points used to scan the virtual slide image are used to calculate a best fit surface for the virtual slide image. The distance of each focus point from the best fit surface is then calculated and the largest distance is compared to a predetermined value. If the largest distance from a focus point to the best fit surface is larger than the predetermined value, then the virtual slide image is designated as needing a manual inspection and possible re-scan.

Growing community of inventors

San Diego, CA, United States of America

Kiran Saligrama