Columbia University

Massachusetts Institute of Technology

James O'Sullivan

Nima Mesgarani

Zhuo Chen

Yi Luo

Thomas Francis Quatieri, Jr

Christopher J Smalt

Michael S Brandstein

Paul T Calamia

Gregory Alan Ciccarelli

Joseph Perricone

Michael Nolan

Stephanie Haro

Disclosed are devices, systems, apparatus, methods, products, and other implementations, including a method comprising obtaining, by a device, a combined sound signal for signals combined from multiple sound sources in an area in which a person is located, and applying, by the device, speech-separation processing (e.g., deep attractor network (DAN) processing, online DAN processing, LSTM-TasNet processing, Conv-TasNet processing), to the combined sound signal from the multiple sound sources to derive a plurality of separated signals that each contains signals corresponding to different groups of the multiple sound sources. The method further includes obtaining, by the device, neural signals for the person, the neural signals being indicative of one or more of the multiple sound sources the person is attentive to, and selecting one of the plurality of separated signals based on the obtained neural signals. The selected signal may then be processed (amplified, attenuated).

Systems and methods for speech separation and neural decoding of attentional selection in multi-speaker environments

In one aspect of the present disclosure, method includes: receiving neural data responsive to a listener's auditory attention; receiving an acoustic signal responsive to a plurality of acoustic sources; for each of the plurality of acoustic sources: generating, from the received acoustic signal, audio data comprising one or more features of the acoustic source, forming combined data representative of the neural data and the audio data, and providing the combined data to a classification network configured to calculate a similarity score between the neural data and the acoustic source using one or more similarity metrics; and using the similarity scores calculated for each of the acoustic sources to identify, from the plurality of acoustic sources, an acoustic source associated with the listener's auditory attention.

End-to-end deep neural network for auditory attention decoding

Growing community of inventors

New York, NY, United States of America

James O'Sullivan