Koninklijke Philips Corporation N.V.

Vivek Varma Datla

Sheikh Sadid Al Hasan

Oladimeji Feyisetan Farri

Junyi Liu

Ashequl Qadir

Aaditya Prakash

Kathy Mi Young Lee

Tilak Raj Arora

Yuan Ling

Adi Prakash

Tao Li

Reza Ghaeini

Rithesh Sreenivasan

Kathy Lee

Siyuan Zhao

Techniques are described for training and/or utilizing sub-agent machine learning models to generate candidate dialog responses. In various implementations, a user-facing dialog agent (), or another component on its behalf, selects one of the candidate responses which is closest to user defined global priority objectives (). Global priority objectives can include values () for a variety of dialog features such as emotion, confusion, objective-relatedness, personality, verbosity, etc. In various implementations, each machine learning model includes an encoder portion and a decoder portion. Each encoder portion and decoder portion can be a recurrent neural network (RNN) model, such as a RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer.

System for multi-perspective discourse within a dialog

An augmented reality (AR) content generation method includes: acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device; receiving, from a microphone of the AR device, a triggering audio segment; generating one or more query data structures from both the one or more images and the triggering audio segment; retrieving AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query data structures from a database; and outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device.

Systems and methods for augmented reality-enhanced field services support

A system () for automated question answering, including: semantic space () generated from a corpus of questions and answers; a user interface () configured to receive a question; and a processor () comprising: (i) a question decomposition engine () configured to decompose the question into a domain, a keyword, and a focus word; (ii) a question similarity generator () configured to identify one or more questions in a semantic space using the decomposed question; (iii) an answer extraction and ranking engine () configured to: extract, from the semantic space, answers associated with the one or more identified questions; and identify one or more of the extracted answers as a best answer; and (iv) an answer tuning engine () configured to fine-tune the identified best answer using one or more of the domain, keyword, and focus word; wherein the fine-tuned answer is provided to the user via the user interface.

Multi domain real-time question answering system

A method for determining the answer to a query in a document, including: encoding, by an encoder, the query and the document; generating a query-aware context encodings G by a bidirectional attention system using the encoded query and the encoded document; performing a hierarchical self-attention on the query aware document by a hierarchical self-attention system by applying a word to word attention and a word to sentence attention mechanism resulting in a matrix M; and determining the starting word and the ending word of the answer in the document by a span detector based upon the matrix M.

Hierarchical self-attention for machine comprehension

Techniques are described herein for drawing conclusions using free form texts and external resources. In various embodiments, free form input data () may be segmented () into a plurality of input data segments. A first input data segment may be compared () with an external resource () to identify a first candidate conclusion. A reinforcement learning trained agent () may be applied () to make a first determination of whether to accept or reject the first candidate conclusion. Similar actions may be performed with a second input data segment to make a second determination of whether to accept or reject a second candidate conclusion. A final conclusion may be presented () based on the first and second determinations of the reinforcement learning trained agent with respect to at least the first candidate conclusion and the second candidate conclusion.

Drawing conclusions from free form texts with deep reinforcement learning

Techniques are described herein for training and applying memory neural networks, such as 'condensed' memory neural networks ('C-MemNN') and/or &#x201c;average&#x201d; memory neural networks (&#x201c;A-MemNN&#x201d;). In various embodiments, the memory neural networks may be iteratively trained using training data in the form of free form clinical notes and clinical reference documents. In various embodiments, during each iteration of the training, a so-called &#x201c;condensed&#x201d; memory state may be generated and used as part of the next iteration. Once trained, a free form clinical note associated with a patient may be applied as input across the memory neural network to predict one or more diagnoses or outcomes of the patient.

Condensed memory networks

The described embodiments relate to systems, methods, and apparatus for providing a multimodal deep memory network () capable of generating patient diagnoses (). The multimodal deep memory network can employ different neural networks, such as a recurrent neural network and a convolution neural network, for creating embeddings () from medical images () and electronic health records (). Connections between the input embeddings () and diagnoses embeddings () can be based on an amount of attention that was given to the images and electronic health records when creating a particular diagnosis. For instance, the amount of attention can be characterized by data () that is generated based on sensors that monitor eye movements of clinicians observing the medical images and electronic health records. Resulting patient diagnoses can be provided according to a predetermined classification of weights, or a compilation of words that are generated over multiple iterations of the multimodal deep memory network.

Systems, methods, and apparatus for diagnostic inferencing with a multimodal deep memory network

A method for determining, from a document, an answer to a query using a query answering system, comprising: (i) encoding, using an encoder, one or more documents; (ii) encoding a received query; (iii) generating, using an attention mechanism, a query-aware document representation comprising alignment between one or more words in one of the plurality of documents and one or more words in the query; (iv) generating, using a hierarchical self-attention mechanism, a word-to-sentence alignment of the query-aware document representation; (v) labeling, using a conditional random field classifier, each of a plurality of words in the word-to-sentence alignment with one of a one of a plurality of different sequence identifiers, resulting in possible labeled answering spans; and (vi) generating, from the one or more possible labeled answering spans, a response to the query.

CRF-based span prediction for fine machine learning comprehension

A medical information retrieval system comprises a natural language processing system that processes a vocal user query to identify key words and phrases. These key words and phrases are provided to an inferencing engine that provides a set of knowledge-based inferences from medical knowledge sources, based on these key words and phrases. Thereafter, these knowledge-based inferences are provided to an information retrieval engine that retrieves a corresponding plurality of medical articles based on these knowledge-based inferences, and ranks each with respect to the knowledge-based inferences. A summary engine receives the ranked articles and creates a model based on the topical keywords and candidate sentences found in the highly ranked articles. A paraphrase engine processes the candidate sentences to provide a summary response based on a knowledge-based paraphrase model. An audio output device renders the summary report as the response to the user's original vocal query.

Patient-centric clinical knowledge discovery system

Techniques described herein relate to semi-supervised training and application of stacked autoencoders and other classifiers for predictive and other purposes. In various embodiments, a semi-supervised model () may be trained for sentence classification, and may combine what is referred to herein as a 'residual stacked de-noising autoencoder' ('RSDA') (), which may be unsupervised, with a supervised classifier () such as a classification neural network (e.g., a multilayer perceptron, or &#x201c;MLP&#x201d;). In various embodiments, the RSDA may be a stacked denoising autoencoder that may or may not include one or more residual connections. If present, the residual connections may help the RSDA &#x201c;remember&#x201d; forgotten information across multiple layers. In various embodiments, the semi-supervised model may be trained with unlabeled data (for the RSDA) and labeled data (for the classifier) simultaneously.

Semi-supervised classification with stacked autoencoder

Methods and systems for generating text from a haptic-based input. The system may include an interface for receiving a haptic-based input and a processor executing instructions stored on a memory and providing a model. The model is configured to at least receive the haptic-based input and supply a text describing the haptic-based input using the interface.

Haptic input text generation

Various embodiments described herein relate to a method, system, and non-transitory machine-readable medium including one or more of the following: extracting a first concept from input data presented for processing by a downstream function; identifying external data from an external resource based on the first concept; extracting a second concept from the external data; revising the first concept based on the second concept to produce a revised concept, wherein revising includes: applying a machine learning agent to determine whether to keep the first concept or adopt the second concept, and adopting the second concept in place of the first concept for use as the revised concept based on a decision by the machine learning agent to adopt the second concept; and further processing the revised concept according to the downstream function to generate an output.

Extracted concept normalization using external evidence

Methods and systems for generating a question from free text. The system is trained on a corpus of data and receives a tuple consisting of a paragraph (free text), a focused fact, and a question type. The system implements a language model to find the most optimal combination of words to return a question for the paragraph about the focused fact.

Question generation

A system for automated question answering, comprising: a user interface configured to receive a query from a user; a question decomposition engine configured to decompose the query into one or more sub-questions and one or more contexts, and to align the sub-questions with contexts to generate question-context pairs; a query engine configured to query one or more answer resources with the question-context pairs to identify information likely to comprise an answer; and an answer generator configured to: (i) generate question-context-answer triples using the identified information from the query engine; (ii) select a generated question-context-answer triple comprising information most likely to comprise an answer to the identified sub-question; (iii) extract from the selected question-context-answer triple a portion of the associated information comprising an answer to the identified sub-question; and (iv) generate a natural language answer comprising a response to the query posed by the user.

Growing community of inventors

Ashland, MA, United States of America

Vivek Varma Datla