Microsoft Technology Licensing, LLC

Tian Wang

Kazuhito Koishida

Hosam Adel Khalil

Wei-ge Chen

Xiaoqin Sun

Vladimir Cuperman

Allen Gersho

Guo-Wei Shieh

Wei-Ge Chen

Mu Han

Feng Zhuge

A method of compensating for jitter in a packet stream is described. The method comprises placing undecoded frames extracted from packets in the packet stream into a jitter buffer while decoding frames from the jitter buffer and placing the decoded frames into a sample buffer at a rate determined using an average playout delay. The average playout delay is the running average of the playout delay calculated for each packet as each packet becomes available. The playout delay for each packet is the sum of a sample buffer delay and a jitter buffer delay. As each packet is received, the average playout delay is adjusted based on a comparison of the playout delay associated with the received packet to the current average playout delay.

Network jitter smoothing with reduced delay

Techniques and tools related to delayed or lost coded audio information are described. For example, a concealment technique for one or more missing frames is selected based on one or more factors that include a classification of each of one or more available frames near the one or more missing frames. As another example, information from a concealment signal is used to produce substitute information that is relied on in decoding a subsequent frame. As yet another example, a data structure having nodes corresponding to received packet delays is used to determine a desired decoder packet delay value.

Robust decoder

Techniques and tools related to coding and decoding of audio information are described. For example, redundant coded information for decoding a current frame includes signal history information associated with only a portion of a previous frame. As another example, redundant coded information for decoding a coded unit includes parameters for a codebook stage to be used in decoding the current coded unit only if the previous coded unit is not available. As yet another example, coded audio units each include a field indicating whether the coded unit includes main encoded information representing a segment of an audio signal, and whether the coded unit includes redundant coded information for use in decoding main encoded information.

Sub-band voice codec with multi-stage codebooks and redundant coding

Techniques and tools are described for processing reconstructed audio signals. For example, a reconstructed audio signal is filtered in the time domain using filter coefficients that are calculated, at least in part, in the frequency domain. As another example, producing a set of filter coefficients for filtering a reconstructed audio signal includes clipping one or more peaks of a set of coefficient values. As yet another example, for a sub-band codec, in a frequency region near an intersection between two sub-bands, a reconstructed composite signal is enhanced.

Audio codec post-filter

Various strategies for rate/quality control and loss resiliency in an audio codec are described. The various strategies can be used in combination or independently. For example, a real-time speech codec uses intra frame coding/decoding, adaptive multi-mode forward error correction ['FEC'], and rate/quality control techniques. Intra frames help a decoder recover quickly from packet losses, while compression efficiency is still emphasized with predicted frames. Various strategies for inserting intra frames and signaling intra/predicted frames are described. With the adaptive multi-mode FEC, an encoder adaptively selects between multiple modes to efficiently and quickly provide a level of FEC that takes into account the bandwidth currently available for FEC. The FEC information itself may be predictively encoded and decoded relative to primary encoded information. Various rate/quality and FEC control strategies allow additional adaptation to available bandwidth and network conditions.

Audio encoding and decoding with intra frames and adaptive forward error correction

A gain-constrained noise suppression for speech more precisely estimates noise, including during speech, to reduce musical noise artifacts introduced from noise suppression. The noise suppression operates by applying a spectral gain G(m, k) to each short-time spectrum value S(m, k) of a speech signal, where m is the frame number and k is the spectrum index. The spectrum values are grouped into frequency bins, and a noise characteristic estimated for each bin classified as a 'noise bin.' An energy parameter is smoothed in both the time domain and the frequency domain to improve noise estimation per bin. The gain factors G(m, k) are calculated based on the current signal spectrum and the noise estimation, then smoothed before being applied to the signal spectral values S(m, k). First, a noisy factor is computed based on a ratio of the number of noise bins to the total number of bins for the current frame, where a zero-valued noisy factor means only using constant gain for all the spectrum values and noisy factor of one means no smoothing at all. Then, this noisy factor is used to alter the gain factors, such as by cutting off the high frequency components of the gain factors in the frequency domain.

Gain constrained noise suppression

An enhanced low-bit rate parametric voice coder that groups a number of frames from an underlying frame-based vocoder, such as MELP, into a superframe structure. Parameters are extracted from the group of underlying frames and quantized into the superframe which allows the bit rate of the underlying coding to be reduced without increasing the distortion. The speech data coded in the superframe structure can then be directly synthesized to speech or may be transcoded to a format so that an underlying frame-based vocoder performs the synthesis. The superframe structure includes additional error detection and correction data to reduce the distortion caused by the communication of bit errors.

LPC-harmonic vocoder with superframe structure

An efficient and accurate classification method for classifying speech and music signals, or other diverse signal types, is provided. The method and system are especially, although not exclusively, suited for use in real-time applications. Long-term and short-term features are extracted relative to each frame, whereby short-term features are used to detect a potential switching point at which to switch a coder operating mode, and long-term features are used to classify each frame and validate the potential switch at the potential switch point according to the classification and a predefined criterion.

Growing community of inventors

Redmond, WA, United States of America

Tian Wang