Koninklijke Philips Corporation N.V.

Junyi Liu

Oladimeji Feyisetan Farri

Sheikh Sadid Al Hasan

Vivek Varma Datla

Ashequl Qadir

Aaditya Prakash

Kathy Mi Young Lee

Yuan Ling

Tilak Raj Arora

Adi Prakash

Xianshu Zhu

Reza Ghaeini

Rithesh Sreenivasan

Bo Liu

Shreya Anand

Siyuan Zhao

Kathy Lee

Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (). Global priority objectives can include values () for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

System for multi-perspective discourse within a dialog

A system () for automated question answering, including: semantic space () generated from a corpus of questions and answers; a user interface () configured to receive a question; and a processor () comprising: (i) a question decomposition engine () configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator () configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine () configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine () configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Multi domain real-time question answering system

Techniques are described herein for drawing conclusions using free form texts and external resources. In various embodiments, free form input data () may be segmented () into a plurality of input data segments. A first input data segment may be compared () with an external resource () to identify a first candidate conclusion. A reinforcement learning trained agent () may be applied () to make a first determination of whether to accept or reject the first candidate conclusion. Similar actions may be performed with a second input data segment to make a second determination of whether to accept or reject a second candidate conclusion. A final conclusion may be presented () based on the first and second determinations of the reinforcement learning trained agent with respect to at least the first candidate conclusion and the second candidate conclusion.

Drawing conclusions from free form texts with deep reinforcement learning

Techniques are described herein for training and applying memory neural networks, such as 'condensed' memory neural networks ('C-MemNN') and/or &#x201c;average&#x201d; memory neural networks (&#x201c;A-MemNN&#x201d;). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called &#x201c;condensed&#x201d; memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Condensed memory networks

The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network () capable of generating patient diagnoses (). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings () from medical images () and electronic health records (). Connections between the input embeddings () and diagnoses embeddings () can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data () that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records. Resulting patient diagnoses can be provided according to a predetermined classification of weights, or a compilation of words that are generated over multiple iterations of the multimodal deep memory network.

Systems, methods, and apparatus for diagnostic inferencing with a multimodal deep memory network

A medical information retrieval system comprises a natural language processing system that processes a vocal user query to identify key words and phrases. These key words and phrases are provided to an inferencing engine that provides a set of knowledge-based inferences from medical knowledge sources, based on these key words and phrases. Thereafter, these knowledge-based inferences are provided to an information retrieval engine that retrieves a corresponding plurality of medical articles based on these knowledge-based inferences, and ranks each with respect to the knowledge-based inferences. A summary engine receives the ranked articles and creates a model based on the topical keywords and candidate sentences found in the highly ranked articles. A paraphrase engine processes the candidate sentences to provide a summary response based on a knowledge-based paraphrase model. An audio output device renders the summary report as the response to the user's original vocal query.

Patient-centric clinical knowledge discovery system

Techniques described herein relate to semi-supervised training and application of stacked autoencoders and other classifiers for predictive and other purposes. In various embodiments, a semi-supervised model () may be trained for sentence classification, and may combine what is referred to herein as a 'residual stacked de-noising autoencoder' ('RSDA') (), which may be unsupervised, with a supervised classifier () such as a classification neural network (e.g., a multilayer perceptron, or &#x201c;MLP&#x201d;). In various embodiments, the RSDA may be a stacked denoising autoencoder that may or may not include one or more residual connections. If present, the residual connections may help the RSDA &#x201c;remember&#x201d; forgotten information across multiple layers. In various embodiments, the semi-supervised model may be trained with unlabeled data (for the RSDA) and labeled data (for the classifier) simultaneously.

Semi-supervised classification with stacked autoencoder

Methods and systems for generating text from a haptic-based input. The system may include an interface for receiving a haptic-based input and a processor executing instructions stored on a memory and providing a model. The model is configured to at least receive the haptic-based input and supply a text describing the haptic-based input using the interface.

Haptic input text generation

Techniques are provided for generating and applying a granular attention hierarchical neural network model to classify a document. In various embodiments, data indicative of the document may be obtained () and processed () into a first layer of two or more layers of a hierarchical network model using a dual granularity attention mechanism to generate first layer output data, wherein the dual granularity attention mechanism weighs some portions of the data indicative of the document more heavily. Some portions of the data indicative of the document are integrated into the hieratical network model during training of the dual granularity attention mechanism. The first layer output data may be processed () in the second of two or more layers of the hierarchical network model to generate second layer output data. A classification label can be generated () from the second layer output data.

Hierarchical neural networks with granularized attention

Methods and systems for generating a question from free text. The system is trained on a corpus of data and receives a tuple consisting of a paragraph (free text), a focused fact, and a question type. The system implements a language model to find the most optimal combination of words to return a question for the paragraph about the focused fact.

Question generation

A system, method and device for determining and notifying a clinician of information relevant to the clinician. The method that is performed by the device or system includes identifying at least one keyword in a user profile of a clinician, identifying at least one content word in a new information item, determining a relevance score between the new information item and the clinician based on the at least one keyword and the at least one content word and when the relevance score is above a predetermined threshold value, generating a notification for the clinician indicating the new information item.

Device, system, and method for determining information relevant to a clinician

When performing semantic search for related clinical concepts based on their semantic meaning, a semantic search engine framework employs a client web interface () (e.g., a browser) that receives keyword search information (), and a web server () that communicates with a semantic analysis engine (). A natural language processing (NLP) engine (server) () receives and stores clinical notes and information (), extracts clinical concepts from the clinical notes and stores them in a NoSQL database (). The NLP engine converts unstructured free text notes into structured actionable data. The semantic analysis engine analyses user-entered keywords, maps them to a UMLS concept, and identifies related concepts based on one or more relational knowledge sources. The web server searches for the related concept IDs in the NoSQL database and returns a list () of relevant notes and concepts for display to, and selection by, the user.

Systems and methods for semantic search and extraction of related concepts from clinical documents

A system for automated question answering, comprising: a user interface configured to receive a query from a user; a question decomposition engine configured to decompose the query into one or more sub-questions and one or more contexts, and to align the sub-questions with contexts to generate question-context pairs; a query engine configured to query one or more answer resources with the question-context pairs to identify information likely to comprise an answer; and an answer generator configured to: (i) generate question-context-answer triples using the identified information from the query engine; (ii) select a generated question-context-answer triple comprising information most likely to comprise an answer to the identified sub-question; (iii) extract from the selected question-context-answer triple a portion of the associated information comprising an answer to the identified sub-question; and (iv) generate a natural language answer comprising a response to the query posed by the user.

Open domain real-time question answering

The present disclosure pertains to a paraphrase generation system. The system comprises one or more hardware processors and/or other components. The system is configured to obtain a training corpus. The training corpus comprises language and known paraphrases of the language. The system is configured to generate, based on the training corpus, a word-level attention-based model and a character-level attention-based model. The system is configured to provide one or more candidate paraphrases of a natural language input based on both the word-level and character-level attention-based models. The word-level attention-based model is a word-level bidirectional long short term memory (LSTM) network and the character-level attention-based model is a character-level bidirectional LSTM network. The word-level and character level LSTM networks are generated based on words and characters in the training corpus. In some embodiments, the LSTM networks are stacked residual LSTM networks comprising residual connections between stacked layers of a given LSTM network.

Growing community of inventors

Windham, NH, United States of America

Junyi Liu