Adobe, Inc.

Amazon Technologies, Inc.

Brian John King

Paris Smaragdis

Gautham J Mysore

Bjorn Hoffmeister

Roland Maas

Sree Hari Krishnan Parthasarathi

Siegfried Kunzmann

Athanasios Mouchtaris

Charles Van Winkle

Feng-Ju Chang

Maurizio Omologo

Martin Radfar

A multi-channel transformer acoustic model that processes a plurality of audio signals output by microphones of a microphone array and outputs probabilities for acoustic units of an utterance represented in the audio signals. The audio signals represent the individual microphones' respective capturing of the utterance. The multi-channel model may perform self-attention on embeddings of the audio signals and then cross-channel attention across the attended audio signals. The cross-channel attention may involve processing of signals relative to each other to model the relationships across channels within and across time frames. The multi-channel model may include a transducer to perform processing frame-by-frame.

Automatic speech recognition

A system configured to process speech commands may classify incoming audio as desired speech, undesired speech, or non-speech. Desired speech is speech that is from a same speaker as reference speech. The reference speech may be obtained from a configuration session or from a first portion of input speech that includes a wakeword. The reference speech may be encoded using a recurrent neural network (RNN) encoder to create a reference feature vector. The reference feature vector and incoming audio data may be processed by a trained neural network classifier to label the incoming audio data (for example, frame-by-frame) as to whether each frame is spoken by the same speaker as the reference speech. The labels may be passed to an automatic speech recognition (ASR) component which may allow the ASR component to focus its processing on the desired speech.

Anchored speech detection and speech recognition

Time interval sound alignment techniques are described. In one or more implementations, one or more inputs are received via interaction with a user interface that indicate that a first time interval in a first representation of sound data generated from a first sound signal corresponds to a second time interval in a second representation of sound data generated from a second sound signal. A stretch value is calculated based on an amount of time represented in the first and second time intervals, respectively. Aligned sound data is generated from the sound data for the first and second time intervals based on the calculated stretch value.

Time interval sound alignment

Sound rate modification techniques are described. In one or more implementations, an indication is received of an amount that a rate of output of sound data is to be modified. One or more sound rate rules are applied to the sound data that, along with the received indication, are usable to calculate different rates at which different portions of the sound data are to be modified, respectively. The sound data is then output such that the calculated rates are applied.

Sound rate modification

Sound feature priority alignment techniques are described. In one or more implementations, features of sound data are identified from a plurality of recordings. Values are calculated for frames of the sound data from the plurality of recordings. The values are based on similarity of the frames of the sound data from the plurality of recordings to each other, the similarity based on the identified features and a priority that is assigned based on the identified features of respective frames. The sound data from the plurality of recordings is then aligned based at least in part on the calculated values.

Sound feature priority alignment

Sound alignment techniques that employ timing information are described. In one or more implementations, features and timing information of sound data generated from a first sound signal are identified and used to identify features of sound data generated from a second sound signal. The identified features may then be utilized to align portions of the sound data from the first and second sound signals to each other.

Sound alignment using timing information

Sound alignment user interface techniques are described. In one or more implementations, a user interface is output having a first representation of sound data generated from a first sound signal and a second representation of sound data generated from a second sound signal. One or more inputs are received, via interaction with the user interface, that indicate that a first point in time in the first representation corresponds to a second point in time in the second representation. Aligned sound data is generated from the sound data from the first and second sound signals based at least in part on correspondence of the first point in time in the sound data generated from the first sound signal to the second point in time in the sound data generated from the second sound signal.

Sound alignment user interface

Noise robust template matching may be performed. First features of a first signal may be computed. Based at least on a portion of the first features, second features of a second signal may be computed. A new signal may be generated based on at least another portion of the first features and on at least a portion of the second features.

Noise-robust template matching

This specification describes technologies relating to editing audio data. In general, one aspect of the subject matter described in this specification can be embodied in methods that include receiving an audio signal including digital audio data; receiving an input identifying particular audio data of the audio signal corresponding to a noise pulse; and replacing the audio data corresponding to the detected noise pulse using interpolation of adjacent audio data to generate an edited audio signal. Other embodiments of this aspect include corresponding systems, apparatus, and computer program products.

GSM noise removal

This specification describes technologies relating to editing digital audio data. In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving an audio signal including audio data in multiple channels; identifying noise in the audio signal including identifying panning information for the audio data in the signal at each of multiple frequency bands; and attenuating the audio data at one or more frequency bands to generate an edited audio signal when the panning exceeds a specified threshold for each of the one or more frequency bands. Other embodiments of this aspect include corresponding systems, apparatus, and computer program products.

Growing community of inventors

Seattle, WA, United States of America

Brian John King