Columbia University

Sarnoff Corporation

Optios, Inc.

Other

Neuromatters, LLC

Paul Sajda

Lucas Cristobal Parra

Clay Douglas Spence

Jacek Dmochowski

David Jangraw

Paul DeGuzman

Steven Lamont Miller

Shih-Fu Chang

Suhas E Chelian

Jun Wang

John C Pearson

Will McBurnett

Amy Kruse

Eric Pohlmeyer

Thomas F Nugent, Iii

Barbara Hanna

Steve I Hsu

Jordan Muraskin

Jane Asmuth

Jeffrey W Berger

Victor Shih

Jason Sherwin

Daniel Rosenthal

Charles Phillip Cloud

David Bach

Sonakshi Bose Roy

Sameer Saproo

Xueqing Liu

Michael Chase Murdock

To identify physiological states that are predictive of a person's performance, a system provides physiological and behavioral interfaces and a data processing pipeline. Physiological sensors generate physiological data about the person while performing a task. The behavioral interface generates performance data about the person while performing the task. The pipeline collects the physiological and performance data along with reference data from a population of people performing the same or similar tasks. In various implementations, the physiological states are brain states. In one implementation, the pipeline computes bandpower ratios. In another implementation, the pipeline decomposes the physiological data into frequency-banded components, identifies brain states derived from the decomposed data—for example, clusters of correlations of decomposed data envelopes—grades the performance data, compares the graded performance data to the brain states, and identifies statistical relationships between the brain states and levels of performance.

Identifying and strengthening physiological/neurophysiological states predictive of superior performance

The present subject matter relates to techniques for hierarchical deep transcoding. The disclosed system can include a processor that can be configured to receive a functional magnetic resonance imaging (fMRI) data and/or an extracranial electroencephalogram (EEG) data and reconstruct a latent source space from the fMRI data and/or the EEG data by decoding the EEG data and/or the fMRI data to a latent source space. The fMRI data and the EEG data can be simultaneously acquired.

Systems and methods for fusing EEG and fMRI through hierarchical deep transcoding

The present disclosure relates to systems and methods for providing a hybrid brain-computer-interface (hBCI) that can detect an individual's reinforcement signals (e.g., level of interest, arousal, emotional reactivity, cognitive fatigue, cognitive state, or the like) in and/or response to objects, events, and/or actions in an environment by generating reinforcement signals for improving an AI agent controlling the environment, such as an autonomous vehicle. Although the disclosed subject matter is discussed within the context of an autonomous vehicle virtual reality game in the exemplary embodiments of the present disclosure, the disclosed system can be applicable to any other environment in which the human user's sensory input is to be used to influence actions within the environment. Furthermore, the systems and methods disclosed can use neural, physiological, or behavioral signatures to inform deep reinforcement learning based AI systems to enhance user comfort and trust in automation.

Systems and methods for deep reinforcement learning using a brain-artificial intelligence interface

To identify physiological states that are predictive of a person's performance, a system provides physiological and behavioral interfaces and a data processing pipeline. Physiological sensors generate physiological data about the person while performing a task. The behavioral interface generates performance data about the person while performing the task. The pipeline collects the physiological and performance data along with reference data from a population of people performing the same or similar tasks. In various implementations, the physiological states are brain states. In one implementation, the pipeline computes bandpower ratios. In another implementation, the pipeline decomposes the physiological data into frequency-banded components, identifies brain states derived from the decomposed data&#x2014;for example, clusters of correlations of decomposed data envelopes&#x2014;grades the performance data, compares the graded performance data to the brain states, and identifies statistical relationships between the brain states and levels of performance.

Methods are disclosed for determining an efficacy of a stimulus based on one or more measurable physiological responses to one or more stimuli including one or more stimulus features. Data is acquired on physiological responses of a group of one or more subjects to presentation of one or more stimuli including one or more stimulus features. The data on the one or more physiological responses of the one or more subjects is correlated with the presentation of the one or more stimulus features included in the one or more stimuli. The correlated data on the one or more physiological responses are associated with a separately-determined efficacy of the one or more stimuli to form a stimulus efficacy model. From this information, a projected efficacy of a stimulus is determinable by comparing one or more subsequently-measured physiological responses to the stimulus with the stimulus efficacy model.

Method for predicting efficacy of a stimulus by measuring physiological response to stimuli

Methods and systems for evaluating a subject's response to a task related to a stimulus include a brain activity sensor, such as an EEG sensor, for measuring neural data generated by the subject in response to the visual stimulus. One or more neural discriminators can be calculated based on the neural data. In order to generate one or more neural discriminators, two or more task conditions can be selected for discrimination. The subject's performance can be evaluated based on the one or more neural discriminators. Feedback can be provided to the subject to assist in achieving better performance.

Systems and methods for identifying and tracking neural correlates of baseball pitch trajectories

Human visual perception is able to recognize a wide range of targets but has limited throughput. Machine vision can process images at a high speed but suffers from inadequate recognition accuracy of general target classes. Systems and methods are provided that combine the strengths of both systems and improve upon existing multimedia processing systems and methods to provide enhanced multimedia labeling, categorization, searching, and navigation.

Rapid image annotation via brain state decoding and visual pattern mining

An EEG cap () having 64 or 128 electrodes () is placed on the head of the subject () who is viewing CRT monitor (). The signals on each channel are amplified by amplifier () and sent to an analog-to-digital converter (). PC () captures and records the amplified signals and the signals are processed by signal processing PC () performing linear signal processing. The resulting signal is sent back to a feedback/display PC () having monitor ().

Single Trial detection in encephalography

Single trial detection in encephalography

A system and a concomitant method for providing programming content in response to an audio signal. The programming content and the audio signal are transmitted in a network having a forward channel and a back channel. In one embodiment, the system comprises a local processing unit and a remote server computer. A first user provides a first audio signal containing a request for programming content from a service provider. The local processing unit receives the first audio signal and transmits the received first audio signal to a service provider via the back channel. The remote server computer receives the first audio signal from the back channel, recognizes the first user and the request for programming content, retrieves the requested programming content from a program database and transmits the programming content to the local processing unit via the forward channel.

System and method for providing programming content in response to an audio signal

The invention is a method and system for mosaicing images of the eye to create high resolution, wide-field ophthalmic images for the diagnosis and treatment of eye diseases. The inventive method acquires a first image of the eye, acquires a second image of the eye, and processes the images to produce a mosaic representation. The second image includes a portion of the first image. To guide in acquiring the second image, the method may view the first image while acquiring the second image. The images typically are acquired using either a direct ophthalmoscope or a slitlamp biomicroscope. The method further may convert the images to a digital format. The step of processing includes aligning and merging the images, and conducting real-time processing and non-real-time processing. Real-time processing may include eliminating non-overlapping images, image registration, topology inference, local-to-global alignment, image fusion, signal estimation, and multiresolution blending, while the non-real-time processing may include local alignment, global alignment, image registration, intra-alignment, inter-alignment, signal averaging, and photometric blending.

Mosaicing and enhancement of images for ophthalmic diagnosis and documentation

A signal processing apparatus and concomitant method for learning and integrating features from multiple resolutions for detecting and/or classifying objects. The signal processing apparatus comprises a hierarchical pyramid of neural networks (HPNN) having a &ldquo;fine-to-coarse&rdquo; structure or a combination of the &ldquo;fine-to-coarse&rdquo; and the &ldquo;coarse-to-fine&rdquo; structures.

Method and apparatus for training a neural network to detect objects in an image

A method and apparatus for training and operating a neural network using gated data. The neural network is a mixture of experts that performs &ldquo;soft&rdquo; partitioning of a network of experts. In a specific embodiment, the technique is used to detect malignancy by analyzing skin surface potential data. In particular, the invention uses certain patient information, such as menstrual cycle information, to &ldquo;gate&rdquo; the expert output data into particular populations, i.e., the network is soft partitioned into the populations. An Expectation-Maximization (EM) routine is used to train the neural network using known patient information, known measured skin potential data and correct diagnosis for the particular training data and patient information. Once trained, the neural network parameters are used in a classifier for predicting breast cancer malignancy when given the patient information and skin potentials of other patients.

Method and apparatus for training and operating a neural network for detecting breast cancer

A signal processing apparatus and concomitant method for learning and integrating features from multiple resolutions for detecting and/or classifying objects are presented. Neural networks in a pattern tree structure with tree-structured descriptions of objects in terms of simple sub-patterns, are grown and trained to detect and integrate the sub-patterns. A plurality of objective functions and their approximations are presented to train the neural networks to detect sub-patterns of features of some class of objects. Objective functions for training neural networks to detect objects whose positions in the training data are uncertain and for addressing supervised learning where there are potential errors in the training data are also presented.

Growing community of inventors

New York, NY, United States of America

Paul Sajda