Agora Lab, Inc.

Jianyuan Feng

Ruixiang Hang

Bo Wu

Fan Li

Song Li

Jimeng Zheng

Linsheng Zhao

A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session is performed by an electronic communication device. The application is adapted to remove the reverberation of recorded speech signals from far-end participants by the dereverberation approach, render the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generate reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, and combine direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.

System for generating spatial audio with uniform reverberation in real-time communication

A computer-implemented method for generating spatial audio with uniform reverberation in a real-time communication session is performed by a real-time communication software application running on an electronic communication device. The method includes removing the reverberation of recorded speech signals from far-end participants by the dereverberation approach, rendering the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generating reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, combining direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.

System and method for generating spatial audio with uniform reverberation in real-time communication

A new real-time spatial audio rendering system includes a real-time spatial audio rendering computer software application adapted to run on a communication device. The application renders stereo audio from mono audio sources in a virtual room of a listener. The listener can be mobile. The stereo audio is rendered for each listener within the room. The real-time spatial audio rendering system has two different modes, with and without reverberation. Reverberation can provide the sense of the dimensions of the room, First, the anechoic processing module produces the anechoic stereo audio that provides the sense of direction and distance of spatial audio. When reverberation is desired, the reverberation processing module is also performed to provide the sense of the room's dimensions by the spatial audio.

System and method for rendering real-time spatial audio in virtual environment

A method of converting a frame of a voice sample to a singing frame includes obtaining a pitch value of the frame; obtaining formant information of the frame using the pitch value; obtaining aperiodicity information of the frame using the pitch value; obtaining a tonic pitch and chord pitches; using the formant information, the aperiodicity information, the tonic pitch, and the chord pitches to obtain the singing frame; and outputting or saving the singing frame.

Real-time speech to singing conversion

Transforming a voice of a speaker to a reference timbre includes converting a first portion of a source signal of the voice of the speaker into a time-frequency domain to obtain a time-frequency signal; obtaining frequency bin means of magnitudes over time of the time-frequency signal; converting the frequency bin magnitude means into a Bark domain to obtain a source frequency response curve (SR), where SR(i) corresponds to magnitude mean of the ifrequency bin; obtaining respective gains of frequency bins of the Bark domain with respect to a reference frequency response curve (Rf); obtaining equalizer parameters using the respective gains of the frequency bins of the Bark domain; and transforming the first portion to the reference timbre using the equalizer parameters.

Growing community of inventors

Shanghai, China

Jianyuan Feng