Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Corporation

Gordon P Wichern

Jonathan Le Roux

Zhong-Qiu Wang

John R Hershey

Ankush Chakrabarty

Shinji Watanabe

Takaaki Hori

Toshiyuki Hanazawa

Yohei Okato

Niko Moritz

Ryo Aihara

Zhongqiu Wang

Ethan Manilow

Fatemeh Pishdadian

Olga Slizovskaia

Satvik Venkatesh

Aswin Shanmugam Subramanian

John Hershey

A system and a method for detecting anomalous sound are disclosed. The method includes receiving an audio signal from a sound source in a recording environment. The sound source and the recording environment are characterized by a set of attributes including a first attribute pertaining to a first attribute type and a second attribute pertaining to a second attribute type. A multi-head neural network is trained to extract from the received audio signal a first embedding vector indicative of the first attribute type and a second embedding vector indicative of the second attribute type. The first embedding vector is compared with a first set of embedding vectors to classify attributes of the first attribute type and the second embedding vector is compared with a second set of embedding vectors to classify attributes of the second attribute type, to determine a result of anomaly detection.

System and method for unsupervised anomalous sound detection

Embodiments of the present disclosure disclose a system and method for localization of a target sound event. The system collects a first digital representation of an acoustic mixture of sounds of a plurality of sound events, by using an acoustic sensor. The system receives a second digital representation of a sound corresponding to the target sound event. Further, the first digital representation and the second digital representation are processed by a neural network to produce a localization information indicative of a location of an origin of the target sound event with respect to a location of the acoustic sensor.

Method and system for sound event localization and detection

A system and method for reverberation reduction is disclosed. A first Deep Neural Network (DNN) produces a first estimate of a target direct-path signal from a mixture of acoustic signals that include the target direct-path signal and a reverberation of the target direct-path signal. A filter modeling a room impulse response (RIR) for the first estimate is estimated. The filter when applied to the first estimate of the target direct-path signal generates a result closest to a residual between the mixture of the acoustic signals and the first estimate of the target direct-path signal according to a distance function. The estimated filter is used for modeling the RIR.

Method and system for reverberation modeling of speech signals

A system and method for detecting anomalous sound are disclosed. The method includes receiving a spectrogram of an audio signal with elements defined by values in a time-frequency domain of the spectrogram. Each of the values corresponds to an element of the spectrogram that is identified by a coordinate in the time-frequency domain. The time-frequency domain of the spectrogram is partitioned into a context region and a target region. The context region and the target region are processed by a neural network using an attentive neural process to recover values of the spectrogram for elements with coordinates in the target region. The recovered values of the elements of the target region are compared with values of elements of the partitioned target region. An anomaly score is determined based on the comparison. The anomaly score is used for performing a control action.

Method and system for detecting anomalous sound

A speech separation device () of a speech separation system includes a feature amount extraction unit () configured to extract time-series data of a speech feature amount of mixed speech, a block division unit () configured to divide the time-series data of the speech feature amount into blocks having a certain time width, a speech separation neural network () configured to create time-series data of a mask of each of a plurality of speakers from the time-series data of the speech feature amount divided into blocks, and a speech restoration unit () configured to restore the speech data of each of the plurality of speakers from the time-series data of the mask and the time-series data of the speech feature amount of the mixed speech.

Voice separation device, voice separation method, voice separation program, and voice separation system

A system and method for reverberation reduction is disclosed. A first Deep Neural Network (DNN) produces a first estimate of a target direct-path signal from a mixture of acoustic signals that include the target direct-path signal and a reverberation of the target direct-path signal. A filter modeling a room impulse response (RIR) for the first estimate is estimated. The filter when applied to the first estimate of the target direct-path signal generates a result closest to a residual between the mixture of the acoustic signals and the first estimate of the target direct-path signal according to a distance function. A mixture with reduced reverberation of the target direct-path signal is obtained by removing the result of applying the filter to the first estimate of the target direct-path signal from the received mixture. A second DNN produces a second estimate of the target direct-path signal from the mixture with reduced reverberation.

Method and system for dereverberation of speech signals

An audio processing system is provided. The audio processing system comprises an input interface configured to accept an audio signal. Further, the audio processing system comprises a memory configured to store a neural network trained to determine different types of attributes of multiple concurrent audio events of different origins, wherein the types of attributes include time-dependent and time-agnostic attributes of speech and non-speech audio events. Further, the audio processing system comprises a processor configured to process the audio signal with the neural network to produce metadata of the audio signal, the metadata including one or multiple attributes of one or multiple audio events in the audio signal.

System and method for producing metadata of an audio signal

A system for controlling an operation of a machine including a plurality of actuators assisting one or multiple tools to perform one or multiple tasks, in response to receiving an acoustic mixture of signals generated by the tool performing a task and by the plurality of actuators actuating the tool, submit the acoustic mixture of signals into a neural network trained to separate from the acoustic mixture a signal generated by the tool performing the task from signals generated by the actuators actuating the tool to extract the signal generated by the tool performing the task from the acoustic mixture of signals, analyze the extracted signal to produce a state of performance of the task, and execute a control action selected according to the state of performance of the task.

Manufacturing automation using acoustic separation neural network

The audio processing system includes a memory to store a neural network trained to process an audio mixture to output estimation of at least a subset of a set of audio sources present in the audio mixture. The audio sources are subject to hierarchical constraints enforcing a parent-children hierarchy on the set of audio sources, such that a parent audio source in includes a mixture of its one or multiple children audio sources. The subset includes a parent audio source and at least one of its children audio sources. The system further comprises a processor to process a received input audio mixture using the neural network to estimate the subset of audio sources and their mutual relationships according to the parent-children hierarchy. The system further includes an output interface configured to render the extracted audio sources and their mutual relationships.

System and method for hierarchical audio source separation

Systems and methods for audio signal processing including an input interface to receive a noisy audio signal including a mixture of target audio signal and noise. An encoder to map each time-frequency bin of the noisy audio signal to one or more phase-related value from one or more phase quantization codebook of phase-related values indicative of the phase of the target signal. Calculate, for each time-frequency bin of the noisy audio signal, a magnitude ratio value indicative of a ratio of a magnitude of the target audio signal to a magnitude of the noisy audio signal. A filter to cancel the noise from the noisy audio signal based on the phase-related values and the magnitude ratio values to produce an enhanced audio signal. An output interface to output the enhanced audio signal.

Methods and systems for enhancing audio signals corrupted by noise

Systems and methods for an audio signal processing system for transforming an input audio signal. A processor implements steps of a module by inputting an input audio signal into a spectrogram estimator to extract an audio feature sequence, and process the audio feature sequence to output a set of estimated spectrograms. Processing the set of estimated spectrograms and the audio feature sequence using a spectrogram refinement module, to output a set of refined spectrograms. Wherein the processing of the spectrogram refinement module is based on an iterative reconstruction algorithm. Processing the set of refined spectrograms for the one or more target audio signals using a signal refinement module, to obtain the target audio signal estimates. An output interface to output the optimized target audio signal estimates. Wherein the module is optimized by minimizing an error using an optimizer stored in the memory.

Growing community of inventors

Cambridge, MA, United States of America

Gordon P Wichern