Koninklijke Philips Corporation N.v.

Sheikh Sadid Al Hasan

Oladimeji Feyisetan Farri

Junyi Liu

Vivek Varma Datla

Ashequl Qadir

Aaditya Prakash

Yuan Ling

Kathy Mi Young Lee

Tilak Raj Arora

Rithesh Sreenivasan

Christine Menking Swisher

Cristhian Mauricio Potes Blandon

Jonathan Rubin

Tao Li

Reza Ghaeini

Adi Prakash

Bo Liu

Shreya Anand

Xianshu Zhu

Kathy Lee

Siyuan Zhao

Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (). Global priority objectives can include values () for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

System for multi-perspective discourse within a dialog

A method () for generating a textual description of a medical image, comprising: receiving () a medical image of an anatomical region, the image comprising one or more abnormalities; segmenting () the anatomical region in the received medical image from a remainder of the image; identifying () at least one of the one or more abnormalities in the segmented anatomical region; extracting () one or more features from the identified abnormality; generating (), using the extracted features and a trained text generation model, a textual description of the identified abnormality; and reporting (), via a user interface of the system, the generated textual description of the identified abnormality.

Convolutional localization networks for intelligent captioning of medical images

An augmented reality (AR) content generation method includes: acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device; receiving, from a microphone of the AR device, a triggering audio segment; generating one or more query data structures from both the one or more images and the triggering audio segment; retrieving AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query data structures from a database; and outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device.

Systems and methods for augmented reality-enhanced field services support

A system () for automated question answering, including: semantic space () generated from a corpus of questions and answers; a user interface () configured to receive a question; and a processor () comprising: (i) a question decomposition engine () configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator () configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine () configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine () configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Multi domain real-time question answering system

A method for determining the answer to a query in a document, including: encoding, by an encoder, the query and the document; generating a query-aware context encodings G by a bidirectional attention system using the encoded query and the encoded document; performing a hierarchical self-attention on the query aware document by a hierarchical self-attention system by applying a word to word attention and a word to sentence attention mechanism resulting in a matrix M; and determining the starting word and the ending word of the answer in the document by a span detector based upon the matrix M.

Hierarchical self-attention for machine comprehension

Techniques are described herein for drawing conclusions using free form texts and external resources. In various embodiments, free form input data () may be segmented () into a plurality of input data segments. A first input data segment may be compared () with an external resource () to identify a first candidate conclusion. A reinforcement learning trained agent () may be applied () to make a first determination of whether to accept or reject the first candidate conclusion. Similar actions may be performed with a second input data segment to make a second determination of whether to accept or reject a second candidate conclusion. A final conclusion may be presented () based on the first and second determinations of the reinforcement learning trained agent with respect to at least the first candidate conclusion and the second candidate conclusion.

Drawing conclusions from free form texts with deep reinforcement learning

Techniques are described herein for training and applying memory neural networks, such as 'condensed' memory neural networks ('C-MemNN') and/or &#x201c;average&#x201d; memory neural networks (&#x201c;A-MemNN&#x201d;). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called &#x201c;condensed&#x201d; memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Condensed memory networks

The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network () capable of generating patient diagnoses (). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings () from medical images () and electronic health records (). Connections between the input embeddings () and diagnoses embeddings () can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data () that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records. Resulting patient diagnoses can be provided according to a predetermined classification of weights, or a compilation of words that are generated over multiple iterations of the multimodal deep memory network.

Systems, methods, and apparatus for diagnostic inferencing with a multimodal deep memory network

A method for determining, from a document, an answer to a query using a query answering system, comprising: (i) encoding, using an encoder, one or more documents; (ii) encoding a received query; (iii) generating, using an attention mechanism, a query-aware document representation comprising alignment between one or more words in one of the plurality of documents and one or more words in the query; (iv) generating, using a hierarchical self-attention mechanism, a word-to-sentence alignment of the query-aware document representation; (v) labeling, using a conditional random field classifier, each of a plurality of words in the word-to-sentence alignment with one of a one of a plurality of different sequence identifiers, resulting in possible labeled answering spans; and (vi) generating, from the one or more possible labeled answering spans, a response to the query.

CRF-based span prediction for fine machine learning comprehension

A medical information retrieval system comprises a natural language processing system that processes a vocal user query to identify key words and phrases. These key words and phrases are provided to an inferencing engine that provides a set of knowledge-based inferences from medical knowledge sources, based on these key words and phrases. Thereafter, these knowledge-based inferences are provided to an information retrieval engine that retrieves a corresponding plurality of medical articles based on these knowledge-based inferences, and ranks each with respect to the knowledge-based inferences. A summary engine receives the ranked articles and creates a model based on the topical keywords and candidate sentences found in the highly ranked articles. A paraphrase engine processes the candidate sentences to provide a summary response based on a knowledge-based paraphrase model. An audio output device renders the summary report as the response to the user's original vocal query.

Patient-centric clinical knowledge discovery system

Techniques described herein relate to semi-supervised training and application of stacked autoencoders and other classifiers for predictive and other purposes. In various embodiments, a semi-supervised model () may be trained for sentence classification, and may combine what is referred to herein as a 'residual stacked de-noising autoencoder' ('RSDA') (), which may be unsupervised, with a supervised classifier () such as a classification neural network (e.g., a multilayer perceptron, or &#x201c;MLP&#x201d;). In various embodiments, the RSDA may be a stacked denoising autoencoder that may or may not include one or more residual connections. If present, the residual connections may help the RSDA &#x201c;remember&#x201d; forgotten information across multiple layers. In various embodiments, the semi-supervised model may be trained with unlabeled data (for the RSDA) and labeled data (for the classifier) simultaneously.

Semi-supervised classification with stacked autoencoder

Methods and systems for generating text from a haptic-based input. The system may include an interface for receiving a haptic-based input and a processor executing instructions stored on a memory and providing a model. The model is configured to at least receive the haptic-based input and supply a text describing the haptic-based input using the interface.

Haptic input text generation

Techniques are provided for generating and applying a granular attention hierarchical neural network model to classify a document. In various embodiments, data indicative of the document may be obtained () and processed () into a first layer of two or more layers of a hierarchical network model using a dual granularity attention mechanism to generate first layer output data, wherein the dual granularity attention mechanism weighs some portions of the data indicative of the document more heavily. Some portions of the data indicative of the document are integrated into the hieratical network model during training of the dual granularity attention mechanism. The first layer output data may be processed () in the second of two or more layers of the hierarchical network model to generate second layer output data. A classification label can be generated () from the second layer output data.

Growing community of inventors

Cambridge, MA, United States of America

Sheikh Sadid Al Hasan