Amazon Technologies, Inc.

Karim Helwani

Arvindh Krishnaswamy

Mehmet Umut Isik

Jean-Marc Valin

Srikanth Venkata Tenneti

Michael Mark Goodwin

Fangzhou Cheng

Neerad Dilip Phansalkar

Aparna Pandey

Paris Smaragdis

Venkata Snehith Cherukuri

Amin Hani Atrash

Richard Newman

Aarthi Raveendran

Kevin V Macwan

Alexander Caughron

Emmanouil Theodosis

Masahito Togami

Erfan Soltanmohammadi

System and methods are provided for estimating the sound field from partial observations. Estimating an acoustic environment for virtual reality and augmented reality applications is a step in the creation of simulated acoustic sound scenes. In particular, the impulse responses of room can be estimated with a generative model. In a teleconferencing scenario with remote participants and a group of participants in a common physical space, giving the remote participants the impression that all other participants are sitting is in the same room acoustically requires filtering the speech of the remote participants with impulse responses estimated at the desired rendering position in the conference room.

Manifold learning for sound field estimation

Systems, methods, and apparatuses for selecting a model are described. A method includes receiving a request to perform model selection and evaluating multiple models by generating various metrics for each trained model. These metrics include a forewarning time (how much in advance of a failure an alert can be raised), an event recall metric (how many failure events were alerted to in advance), an event precision metric (ratio of true and false positives, and an area under a receiver operating characteristic (ROC) curve. For each trained model, a weighted harmonic mean is calculated from these metrics. A model is then selected based on the calculated means and a report on the selected model is generated and provided.

Prognostics and health management service

A group masked autoencoder may be implemented for anomaly detection. An autoencoder network model may be trained without supervision and applied to output an estimated joint probability distribution of normality for a group of frames of time series data. The estimated joint probability distribution may be used to determine an anomaly score for the time series data. An anomaly may be detected according to the anomaly score and a result that indicates a detected anomaly may be provided.

Group masked autoencoder for anomaly detection

Systems, methods, and apparatuses for detecting anomalies using clusters are described. In some examples, a method includes receiving a request to perform anomaly detection using a plurality of clusters; receiving a data point; determining when the received data point is a part of one of the plurality of clusters utilizing a distance to centers of the one or more clusters, wherein: when the received data point is determined to belong to a normal cluster, assigning the received data point to the determined cluster, updating the cluster, and updating a history for the cluster, when the received data point is determined to belong to an anomalous cluster, raising an anomaly, updating the cluster, and updating a history for the cluster, and when the received data point is determined to not belong to any cluster, raising an anomaly.

Real-time low-complexity stereo speech enhancement with spatial cue preservation may be performed. A stereo speech enhancement system receives a stereo input signal (e.g., a left and right input signal). The stereo speech enhancement system estimates spatial cues for a target speaker and downmixes the stereo input signal into a monaural signal. A low-complexity model may then process the monaural signal to generate an enhanced monaural signal. The stereo speech enhancement system upmixes the enhanced monaural signal based on the estimated spatial cues for the target speaker, to generate an enhanced stereo output signal.

Real-time low-complexity stereo speech enhancement with spatial cue preservation

Techniques for training and using a machine learning model for estimation of reverberation in a multi-task learning framework are described. According to some embodiments, the multi-task learning framework improves the performance of the machine learning model by estimating the amount of reverberation present in an input audio recording as a secondary task to the primary task of generating a clean speech portion of the input audio recording. In one embodiment, a model architecture is selected that takes a noisy reverberant recording as an input and outputs an estimate of a clean (e.g., de-reverberated) signal, an estimate of noise (e.g., background noise), and an estimate of the reverb only portion, with the secondary task of estimating the reverb only portion acting as a regularizer that improves the machine learning model's performance in enhancing the reverberant (e.g., and noisy) input speech.

Speech enhancement machine learning model for estimation of reverberation in a multi-task learning framework

Audio processing may be performed with a convolutional neural network that includes positional embeddings. Audio data may be received at an audio processing system. A convolutional neural network that concatenates frequency-positional embeddings at an input layer may be used to process the audio data. A result of processing the audio data through the convolutional neural network may be used to perform an audio processing task.

Convolutional neural network with positional embeddings for audio processing

Joint noise and echo suppression may be performed for enhancing two-way audio communications. Audio data is captured at a communication device and audio data transmitted to the communication device from another communication device are used as input features to a trained machine learning model that uses the transmitted audio data as a reference signal to eliminate residual echo in the captured audio data when also suppressing noise in the captured audio data.

Joint noise and echo suppression for two-way audio communication enhancement

At a first layer of an echo canceler, a first compensation for a first set of properties of output of an audio capture device of a first communication environment is applied. The first set of properties includes a property resulting from a difference in clock speeds of an audio capture device and an audio rendering device of the first communication environment. At a second layer of the echo canceler, at which output of the first layer is received, a second compensation for a second set of properties of the output of the first layer is applied. The second set of properties includes an echo. Applying the compensations comprises modifying neural network weights. Output of the second layer is transmitted to a second communication environment.

Echo cancellation using neural networks for environments with unsynchronized devices for audio capture and rendering

Post-filtering may be performed for ratio masks as part of audio enhancement. Audio data may be received. A machine learning model may be applied to generate gain values for different spectrum bands of the audio data. The gain values may then be modified using an envelope post-filter according to a monotonically increasing function applied to the gain values to produce modified gain values used to generate an enhanced version of the audio data.

Ratio mask post-filtering for audio enhancement

A system may perform adaptive target presence probability to predict a location of a user (e.g., target) at a given time based on accumulated observations. For example, the system may track a location of the user over time and generate observation data associated with a user profile. The observation data may include a plurality of observations, with a single observation corresponding to a location and time at which the user was detected. The system may apply a clustering algorithm to the observation data to generate probability distributions (e.g., clusters) associated with discrete locations. For example, the system may group observations that are in proximity to each other and separate the groups based on location. Using the probability distribution for an individual cluster, the system may determine a likelihood that the user is present at a location corresponding to the cluster.

Adaptive target presence probability estimation

This disclosure describes techniques for selecting which device to respond to user speech within an environment that includes multiple devices. To enable this selection, the techniques described herein create a model indicating a topology of the devices within the environment. To do so, features associated with the user speech are generated and used to generate multi-dimensional points, each representing a corresponding user utterance. The techniques may then group these multi-dimensional points into clusters before projecting the resulting clusters into two dimensions. The two-dimensional clusters may then be used to generate a model (e.g., a Voronoi diagram) representing regions of the environment, with the centroid of each region being associated with a respective device. When a user makes a subsequent request, the user may be localized within one of the regions and the device associated with that region may be chosen to respond to the user.

Growing community of inventors

Mountain View, CA, United States of America

Karim Helwani