Definiens Ag

Astrazeneca Computational Pathology Gmbh

Definiens Gmbh

Guenter Schmidt

Maria Athelogou

Ralf Schoenmeyer

Nicolas Brieu

Arno Schaepe

Joachim Diebold

Tamara Manuelian

Peter Ellenberg

Olivier Pauly

Ansh Kapil

Jan Martin Lesniak

Johannes Zimmermann

Tobias Wiestler

Simon Lanzmich

Moritz Widmaier

Juergen Klenk

Peter Eschenbacher

Sonja Althammer

Melanie Goerner

Florian Leiss

Owen Feehan

Richard Hudson

Dean Jones

Kai Hartmann

Markus Woischnik

Konstantinos Bobolas

Renate Binnig

Markus Rinecker

A convolutional neural network predicts which regions of a tissue slice would be stained by a first stain by training a model to identify those regions based only on tissue stained by a second stain. Thereafter the first stain need not be used to mark cancerous regions on other tissue slices that are stained with the second stain. The training slice is stained with a first immunohistochemical stain and a second counterstain. A target region of an image of the training slice is identified using image analysis based on the first stain. A set of parameters for associated mathematical operations are optimized to train the model to classify pixels of the image as belonging to the target region based on the second stain but not on the first stain. The trained parameters are stored in a database and applied to other images of tissue not stained with the first stain.

Using a first stain to train a model to predict the region stained by a second stain

A method for indicating how a cancer patient will respond to a predetermined therapy relies on spatial statistical analysis of classes of cell centers in a digital image of tissue of the cancer patient. The cell centers are detected in the image of stained tissue of the cancer patient. For each cell center, an image patch that includes the cell center is extracted from the image. A feature vector is generated based on each image patch using a convolutional neural network. A class is assigned to each cell center based on the feature vector associated with each cell center. A score is computed for the image of tissue by performing spatial statistical analysis based on classes of the cell centers. The score indicates how the cancer patient will respond to the predetermined therapy. The predetermined therapy is recommended to the patient if the score is larger than a predetermined threshold.

Deep learning method for predicting patient response to a therapy

A method that provides a graphical indication of whether a patient will have cancer recurrence uses univariate and bivariate prognostic features that were generated as part of a minimal spanning tree (MST). The method determines the values of first and second features. A first value is measured by detecting objects in an image of tissue from the cancer patient stained with a protein-specific IHC biomarker. A second value is measured using objects marked with an mRNA-specific probe biomarker detected in the tissue. The first feature is the univariate prognostic feature for cancer recurrence in a cohort of cancer patients. A combination of the first and second features is the bivariate prognostic feature for cancer recurrence in the cohort. The first and second features are elements of the MST. Nodes of the MST represent the univariate features, edges represent the bivariate features, and edge weights represent prognostic significance of bivariate features.

Predicting prostate cancer recurrence using a prognostic model that combines immunohistochemical staining and gene expression profiling

A system for generating image-based patient profiles acquires digital images from tissue samples and CT and MRI scans. The system detects objects within the images, measures values related to the detected objects, and displays the measured values in patient profile lists that indicate the normal ranges for measured values. The system indicates which measured values fall outside the normal ranges and navigates the user to the objects in the images associated with the abnormal values when the user selects a measured value in the patient profile. Various risks of the existence of different diseases and the probability of success of specific treatments are displayed on a graphical user interface. The system searches for patterns in the patient data and profile lists that reflect those specific risks and success probabilities. A high probability of disease risk or of the success of a specific treatment is indicated on the graphical user interface.

Image-based patient profiles

A novel cancer scoring tool not only generates a score, but it also generates a confidence number. The tool receives a digital image of tissue of a patient. The tool identifies cell objects in the image and from that determines a first score. The magnitude of this first score is indicative of the severity of cancer in the tissue of the patient. The tool uses an overall false negative rate value and an overall false positive rate value to generate a set of second scores. The rate values are determined from training information. From the second scores, the tool determines the confidence number. The confidence number indicates the confidence the tool has in the first score being correct. The first score and an indication of the confidence number and the digital image are all displayed together on the display of the tool.

Tool that analyzes image data and generates and displays a confidence indicator along with a cancer score

Both object-oriented analysis and the faster pixel-oriented analysis are used to recognize patterns in an image of stained tissue. Object-oriented image analysis is used to segment a small portion of the image into object classes. Then the object class to which each pixel in the remainder of the image most probably belongs is determined using decision trees with pixelwise descriptors. The pixels in the remaining image are assigned object classes without segmenting the remainder of the image into objects. After the small portion is segmented into object classes, characteristics of object classes are determined. The pixelwise descriptors describe which pixels are associated with particular object classes by matching the characteristics of object classes to the comparison between pixels at predetermined offsets. A pixel heat map is generated by giving each pixel the color assigned to the object class that the pixelwise descriptors indicate is most probably associated with that pixel.

Learning pixel visual context from object characteristics to generate rich semantic images

A method for co-registering images of tissue slices stained with different biomarkers displays a first digital image of a first tissue slice on a graphical user interface such that an area of the first image is enclosed by a frame. Then a portion of a second image of a second tissue slice is displayed such that the area of the first image enclosed by the frame is co-registered with the displayed portion of the second image. The displayed portion of the second image has the shape of the frame. The tissue slices are both z slices of a tissue sample taken at corresponding positions in the x and y dimensions. The displayed portion of the second image is shifted in the x and y dimensions to coincide with the area of the first image that is enclosed by the frame as the user shifts the first image under the frame.

Evaluation of co-registered images of differently stained tissue slices

An analysis and visualization system analyzes a digital image of a tissue sample. In the sample, cells of a first type are stained in a first way, and cells of a second type are stained in a second way. The system segments the high-resolution image into first and second objects representing cells of the first and second types, respectively. The system also identifies a region of interest, and divides it into tiles. The system generates, for each tile, a first value and a second value. The first and second values for a tile are indicative of densities of the first and second objects in the tile. From the values, a measured correlation coefficient (CC) value is determined. The system compares the measured CC value to a reference CC value, thereby obtaining a correspondence value. The system then displays the image region along with a visualization of the correspondence value.

Detecting and visualizing correlations between measured correlation values and correlation reference values of a pathway

A method for generating an image-based test improves diagnostic accuracy by iteratively modifying rule sets governing image and data analysis of coregistered image tiles. Digital images of stained tissue slices are divided into tiles, and tiles from different images are coregistered. First image objects are linked to selected pixels of the tiles. First numerical data is generated by measuring the first objects. Each pixel of a heat map aggregates first numerical data from coregistered tiles. Second objects are linked to selected pixels of the heat map. Measuring the second objects generates second numerical data. The method improves how well second numerical data correlates with clinical data of the patient whose tissue is analyzed by modifying the rule sets used to generate the first and second objects and the first and second numerical data. The test is defined by those rule sets that produce the best correlation with the clinical data.

Generating image-based diagnostic tests by optimizing image analysis and data mining of co-registered images

A method for determining whether a test biomarker is a stain for a type of cell component, such as membrane or nucleus, involves performing various segmentation processes on an image of tissue stained with the test biomarker. One segmentation process searches for a first cell component type, and another segmentation process searches for a second cell component type by segmenting only stained pixels. The test biomarker is identified as a stain for each component type if the process identifies the component based only on stained pixels. Whether the test biomarker is a membrane stain or nucleus stain is displayed on a graphical user interface. In addition, the method identifies stained pixels corresponding to a second cell component using pixels determined to correspond to a first cell component. An expression profile for the test biomarker is then displayed that indicates the proportion of stained pixels in each type of cell component.

Biomarker evaluation through image analysis

An image-based biomarker is generated using image features obtained through object-oriented image analysis of medical images. The values of a first subset of image features are measured and weighted. The weighted values of the image features are summed to calculate the magnitude of a first image-based biomarker. The magnitude of the biomarker for each patient is correlated with a clinical endpoint, such as a survival time, that was observed for the patient whose medical images were analyzed. The correlation is displayed on a graphical user interface as a scatter plot. A second subset of image features is selected that belong to a second image-based biomarker such that the magnitudes of the second image-based biomarker for the patients better correlate with the clinical endpoints observed for those patients. The second biomarker can then be used to predict the clinical endpoint of other patients whose clinical endpoints have not yet been observed.

Growing community of inventors

Munich, Germany

Guenter Schmidt