Zoom Video Communications, Inc.

Zoom Communications, Inc.

Yuhui Chen

Oded Gal

Zhaofeng Jia

Jiachuan Deng

Bo Ling

Qiyong Liu

Wei Wang

Huixi Zhao

Gengdai Liu

Yian Zhu

Cheng Lun Hu

Yangbin Zeng

Zhengwei Wei

Example methods and systems provide machine-learning assisted acoustic echo cancellation (AEC). The AEC can be used, as an example, to improve the audio quality for online audio and video conferences. A system according to this disclosure includes a pre-trained, machine-learning, AI model designed to detect, in real time, a unitary voice signal, or a signal representing the speech of a single speaker as opposed to that of multiple speakers. A digital signal processing (DSP) algorithm can then detect the echo state, for example, whether distortion results primarily from an echo. Based on these characteristics, the system can, alternatively and automatically apply either a default mode of AEC to the audio signal, or apply a more aggressive mode of AEC.

Machine-learning assisted acoustic echo cancelation

Methods, systems, and apparatus, including computer programs encoded on computer storage media relate to a method for target speaker extraction. A target speaker extraction system receives an audio frame of an audio signal. A multi-speaker detection model analyzes the audio frame to determine whether the audio frame includes only a single-speaker or multiple speakers. When the audio frame includes only a single-speaker, the system inputs the audio frame to a target speaker VAD model to suppress speech in the audio frame from a non-target speaker based on comparing the audio frame to a voiceprint of a target speaker. When the audio frame includes multiple speakers, the system inputs the audio frame to a speech separation model to separate the voice of the target speaker from a voice mixture in the audio frame.

Target speaker mode

Example methods and systems provide automatic audio equalization for online conferences. A target, ideal, audio frequency response for use in online conferencing audio can be designed and preset for reference, and the equalizer can operate in real time to reach the target frequency response for any input. The equalizer can selectively apply per-band equalization to the audio signal as needed to adjust an energy value for each of multiple frequency bands to produce an output audio signal that that can be, as an example, routed through meeting servers or other infrastructure to other conference participants. The equalization can compensate for conditions local to a speaker that would otherwise adversely affect audio quality.

Automatic audio equalization for online conferences

Various embodiments of an apparatus, method(s), system(s) and computer program product(s) described herein are directed to a Viseme Engine. The Viseme Engine receives audio data associated with a user account. The Viseme Engine predicts at least one viseme that corresponds with a portion of phoneme audio data and identifies one or more facial expression parameters associated with the predicted viseme. The facial expression parameters being applicable to a face model. The Viseme Engine renders the predicted viseme according to the one or more facial expression parameters.

Speech driven talking face

Dynamic adjustment of audio characteristics for enhancing musical sound during a networked conference is disclosed. In an embodiment, a method is provided for sound enhancement performed by a device coupled to a network. The method includes receiving an audio signal to be transmitted over the network, detecting when musical content is present in the audio signal, processing the audio signal to enhance voice characteristics to generate an enhanced audio signal when the musical content is not detected, processing the audio signal to enhance music characteristic to generate the enhanced audio signal when the musical content is detected, and transmitting the enhanced audio signal over the network.

Enhancing musical sound during a networked conference

Methods and apparatus for enhancing musical sound during a networked conference

Growing community of inventors

San Jose, CA, United States of America

Yuhui Chen