Qualcomm Incorporated

Moore Intellectual Property Law, Pllc

Shuhua Zhang

Erik Visser

Lae-Hoon Kim

Yinyi Guo

Sunkuk Moon

Vahid Montazeri

Shankar Thagadur Shivappa

S M Akramus Salehin

Raghuveer Peri

Fatemeh Saki

Dongmei Wang

Sanghyun Chi

Anne Katrin Konertz

Cheng-Yu Hung

Kyungguen Byun

Sharon Kaziunas

Siddhartha Goutham Swaminathan

Minho Jin

Ricardo De Jesus Bernal Castillo

Yongwoo Cho

Juhan Nam

Wenliang Lu

Hannes Pessentheiner

JunCheol Cho

Juncheol Cho

A device to perform speech enhancement includes one or more processors configured to process image data to detect at least one of an emotion, a speaker characteristic, or a noise type. The one or more processors are also configured to generate context data based at least in part on the at least one of the emotion, the speaker characteristic, or the noise type. The one or more processors are further configured to obtain input spectral data based on an input signal. The input signal represents sound that includes speech. The one or more processors are also configured to process, using a multi-encoder transformer, the input spectral data and the context data to generate output spectral data that represents a speech enhanced version of the input signal.

Context-data based speech enhancement

A device includes a memory configured to store untransformed ambisonic coefficients at different time segments. The device includes one or more processors configured to obtain the untransformed ambisonic coefficients at the different time segments, where the untransformed ambisonic coefficients at the different time segments represent a soundfield at the different time segments. The one or more processors are configured to apply one adaptive network, based on a constraint that includes preservation of a spatial direction of one or more audio sources in the soundfield at the different time segments, to the untransformed ambisonic coefficients at the different time segments to generate transformed ambisonic coefficients at the different time segments, wherein the transformed ambisonic coefficients at the different time segments represent a modified soundfield at the different time segments, that was modified based on the constraint. The one or more processors are also configured to apply an additional adaptive network.

Transform ambisonic coefficients using an adaptive network for preserving spatial direction

A device includes one or more processors configured to receive an input audio signal. The one or more processors are also configured to process the input audio signal based on a combined representation of multiple sound sources to generate an output audio signal. The combined representation is used to selectively retain or remove sounds of the multiple sound sources from the input audio signal. The one or more processors are further configured to provide the output audio signal to a second device.

Audio processing using sound source representations

A device includes one or more processors configured to obtain audio signals representing sound captured by at least three microphones and determine spatial audio data based on the audio signals. The one or more processors are further configured to determine a metric indicative of wind noise in the audio signals. The metric is based on a comparison of a first value and a second value. The first value corresponds to an aggregate signal based on the spatial audio data, and the second value corresponds to a differential signal based on the spatial audio data.

Spatial audio wind noise detection

A device to perform speech enhancement includes one or more processors configured to obtain input spectral data based on an input signal. The input signal represents sound that includes speech. The one or more processors are also configured to process, using a multi-encoder transformer, the input spectral data and context data to generate output spectral data that represents a speech enhanced version of the input signal.

Context-based speech enhancement

Systems, methods and computer-readable media are provided for speech enhancement using a hybrid neural network. An example process can include receiving, by a first neural network portion of the hybrid neural network, audio data and reference data, the audio data including speech data, noise data, and echo data; filtering, by the first neural network portion, a portion of the audio data based on adapted coefficients of the first neural network portion, the portion of the audio data including the noise data and/or echo data; based on the filtering, generating, by the first neural network portion, filtered audio data including the speech data and an unfiltered portion of the noise data and/or echo data; and based on the filtered audio data and the reference data, extracting, by a second neural network portion of the hybrid neural network, the speech data from the filtered audio data.

Mixed adaptive and fixed coefficient neural networks for speech enhancement

A device for speech generation includes one or more processors configured to receive one or more control parameters indicating target speech characteristics. The one or more processors are also configured to process, using a multi-encoder, an input representation of speech based on the one or more control parameters to generate encoded data corresponding to an audio signal that represents a version of the speech based on the target speech characteristics.

Synthesized speech generation

A device includes a memory configured to store untransformed ambisonic coefficients at different time segments. The device also includes one or more processors configured to obtain the untransformed ambisonic coefficients at the different time segments, where the untransformed ambisonic coefficients at the different time segments represent a soundfield at the different time segments. The one or more processors are also configured to apply one adaptive network, based on a constraint, to the untransformed ambisonic coefficients at the different time segments to generate transformed ambisonic coefficients at the different time segments, wherein the transformed ambisonic coefficients at the different time segments represent a modified soundfield at the different time segments, that was modified based on the constraint.

Transform ambisonic coefficients using an adaptive network

Methods, systems, and devices for auditory enhancement are described. A device may receive a respective auditory signal at each of a set of microphones, where each auditory signal includes a respective representation of a target auditory component and one or more noise artifacts. The device may identify a directionality associated with a source of the target auditory component (e.g., based on an arrangement of the multiple microphones). The device may determine a distribution function for the target auditory component based at least in part on the directionality associated with the source and on the received plurality of auditory signals. The device may generate an estimate of the target auditory component based at least in part on the distribution function and output the estimate of the target auditory component.

Multiple microphone speech generative networks

Methods, systems, computer-readable media, and apparatuses for signal enhancement are presented. One example of such an apparatus includes a receiver configured to produce a remote speech signal from information carried by a wireless signal; a signal canceller configured to perform a signal cancellation operation on a local speech signal to generate a room response; and a filter configured to filter the remote speech signal according to the room response to produce a filtered speech signal. In this example, the signal cancellation operation is based on the remote speech signal as a reference signal.

Acoustic path modeling for signal enhancement

In a particular aspect, a speech generator includes a signal input configured to receive a first audio signal. The speech generator also includes at least one speech signal processor configured to generate a second audio signal based on information associated with the first audio signal and based further on automatic speech recognition (ASR) data associated with the first audio signal.

Enhanced speech generation

Methods, systems, computer-readable media, and apparatuses for gesture control are presented. One example includes indicating, based on information from a first audio input signal, a presence of an object in proximity to a microphone, and increasing a volume level in response to the indicating.

Acoustic gesture detection for control of a hearable device

A drone system and method. Audio signals are received via one or more microphones positioned relative to a location on a drone and one or more of the audio signals are identified as of interest. Flight characteristics of the drone are then controlled based on the audio signals that are of interest.

Drone flight control

In a particular aspect, an apparatus includes an audio sensor configured to receive an input audio signal. The apparatus also includes speech generative circuitry configured to generate a synthesized audio signal based at least partly on automatic speech recognition (ASR) data associated with the input audio signal and based on one or more parameters indicative of state information associated with the input audio signal.

A method for speech modeling by an electronic device is described. The method includes obtaining a real-time noise reference based on a noisy speech signal. The method also includes obtaining a real-time noise dictionary based on the real-time noise reference. The method further includes obtaining a first speech dictionary and a second speech dictionary. The method additionally includes reducing residual noise based on the real-time noise dictionary and the first speech dictionary to produce a residual noise-suppressed speech signal at a first modeling stage. The method also includes generating a reconstructed speech signal based on the residual noise-suppressed speech signal and the second speech dictionary at a second modeling stage.

Growing community of inventors

San Diego, CA, United States of America

Shuhua Zhang