International Business Machines Corporation

Guillaume Jaume

Antonio Foncubierta Rodriguez

Maria Gabrani

Pushpak Pati

Kevin Thandiackal

Methods and systems are provided for processing different-modality digital images of tissue. The method includes, for each image, detecting biological entities in the image and generating an entity graph comprising entity nodes, representing respective biological entities, interconnected by edges representing interactions between entities represented by the entity nodes. The method also includes selecting, from each image, anchor elements comprising elements corresponding to anchor elements of at least one other image, and generating an anchor graph in which anchor nodes, representing respective anchor elements, are interconnected with entity nodes of the entity graph for the image by edges indicating relations between entity nodes and anchor nodes. The method further includes generating a multimodal graph by interconnecting anchor nodes of the anchor graphs for different images via correspondence edges indicating correspondence between anchor nodes, and processing the multimodal graph to output multimodal data, derived from the plurality of images, for medical evaluation.

Processing multimodal images of tissue for medical evaluation

Computer-implemented methods and systems are provided for interpreting a whole-slide image of a tissue specimen. Such a method includes detecting biological entities in a region of interest independently at each magnification level in a whole-slide image. An entity graph is generated for each magnification level, the entity graph comprising nodes, representing respective biological entities detected at that magnification level, interconnected by edges representing interactions between entities. The method also includes, for each region of interest, generating a hierarchical graph, defining a hierarchy of the entity graphs for that region, in which nodes of different entity graphs are interconnected by hierarchical edges representing hierarchical relations between nodes of the entity graphs. The method further comprises supplying the hierarchical graph for the region to an AI system to produce result data corresponding to a medical evaluation of the tissue specimen represented thereby, and outputting the result data for the tissue specimen.

Interpretation of whole-slide images in digital pathology

A system and process for extracting information from filled form images is described. In one example, the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard-coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

Extracting structured information from a document containing filled form images

A system and process for extracting information from filled form images is described. In one example the claimed invention first extracts textual information and the hierarchy in a blank form. This information is then used to extract and understand the content of filled forms. In this way, the system does not have to analyze from the beginning each filled form. The system is designed so that it remains as generic as possible. The number of hard coded rules in the whole pipeline was minimized to offer an adaptive solution able to address the largest number of forms, with various structures and typography. The system is also created to be integrated as a built-in function in a larger pipeline. The form understanding pipeline could be the starting point of any advanced Natural Language Processing application.

A method of determining a hierarchy of a blank template using an image of the blank template and using the determined hierarchy for providing labels and field values of text lines of a filled form document.

Growing community of inventors

Zurich, Switzerland

Guillaume Jaume