Neumora Therapeutics, Inc.

Tathagata Banerjee

Matthew Edward Kollada

Peter Crocker

Amirsina Torfi

Methods and systems are provided for diagnosing mental health conditions using multiple data modalities. In particular, a trained machine learning model is used for mental health diagnosis, wherein the trained model utilizes a dynamic fusion approach for capturing and preserving interactions as well as timing information between the multiple data modalities.

Multimodal dynamic attention fusion

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a clinical recommendation for medical treatment of a patient. In one aspect a method comprises: receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities; processing the multi-modal data characterizing the patient using a machine learning model, in accordance with values of a set of machine learning model parameters, to generate a patient classification that classifies the patient as being included in a patient category from a set of patient categories; determining an uncertainty measure that characterizes an uncertainty of the patient classification generated by the machine learning model; and generating a clinical recommendation for medical treatment of the patient based on: (i) the patient classification, and (ii) the uncertainty measure that characterizes the uncertainty of the patient classification.

Estimating uncertainty in predictions generated by machine learning models

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an encoder neural network and a decoder neural network. In one aspect, a method comprises: updating current values of a set of encoder parameters and current values of a set of decoder parameters using gradients of a reconstruction loss function that measures an error in a reconstruction of multi-modal data from a training example, wherein: the reconstruction loss function comprises a plurality of scaling factors that each scale a respective term in the reconstruction loss function that measures an error in the reconstruction of a corresponding proper subset of feature dimensions of the multi-modal data from the training example.

Machine learning systems for training encoder and decoder neural networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating multi-modal data archetypes. In one aspect, a method comprises obtaining a plurality of training examples, wherein each training example corresponds to a respective patient and includes multi-modal data, having a plurality of feature dimensions, that characterizes the patient; jointly training an encoder neural network and a decoder neural network on the plurality of training examples; and generating a plurality of multi-modal data archetypes that each correspond to a respective dimension of a latent space, comprising, for each multi-modal data archetype: processing a predefined embedding that represents the corresponding dimension of the latent space using the decoder neural network to generate multi-modal data, having the plurality of feature dimensions, that defines the multi-modal data archetype.

Machine learning systems for generating multi-modal data archetypes

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for classifying a patient. In one aspect, a method comprises: receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities; processing the multi-modal data characterizing the patient using an encoder neural network to generate an embedding of the multi-modal data characterizing the patient; determining a respective classification score for each patient category in a set of patient categories based on the embedding of the multi-modal data characterizing the patient; and classifying the patient as being included in a corresponding patient category from the set of patient categories based on the classification scores.

Growing community of inventors

Waltham, MA, United States of America

Tathagata Banerjee