Cisco Technology, Inc.

Advanced Micro Devices Corporation

Hewlett-packard Development Company, L.p.

Wei-Lien Hsu

Eric Chen

Bjørn Winsvold

Eric Yi-Hua Chen

Pi-Hsin Liu

Wei Li

Yi Liu

Espen Moberg

Fuling Liu

Asbjørn Therkelsen

David C Horton

Frank J Gorishek

Travis Wheatley

Travis Wheatly

Managing noise during an online conference session includes obtaining audio data from an endpoint participating in an online conference session. The audio data is derived from audio captured at the endpoint that includes musical sounds. The audio data is processed to identify a portion of the audio data in which a decibel level of the musical sounds is stable for a period of time. Non-musical noise present, if any, in the audio data with the musical sounds is identified and the non-musical noise is attenuated from the audio data to generate noise-reduced musical audio data. The noise-reduced musical audio data is transmitted for play out at one or more other endpoints participating in the online conference session.

Noise management during an online conference session

Systems, methods, and devices are disclosed for detecting an active speaker in a two-way conference. Real time audio in one or more sub band domains are analyzed according to an echo canceller model. Based on the analyzed real time audio, one or more audio metrics are determined from output from an acoustic echo cancellation linear filter. The one or more audio metrics are weighted based on a priority, and a speaker status is determined based on the weighted one or more audio metrics being analyzed according to an active speaker detection model. For an active speaker status, one or more residual echo or noise is removed from the real time audio based on the one or more audio metrics.

Acoustic echo cancellation based sub band domain active speaker detection for audio and video conferencing applications

This disclosure relates to solutions for eliminating undesired audio artifacts, such as background noises, on an audio channel. A process for implementing the technology can include receiving a set of audio segments, analyzing the segments using a first ML model to identify a first probability of unwanted background noises in the segments, and if the first probability exceeds a threshold, analyzing the segments using a second ML model to determine a second probability that the one or more background features exist in the segments. In some aspects, the process can include attenuating audio artifacts in the segments, if the second probability exceeds a second threshold. In some implementations, dynamic time stretching and shrinking can be applied to the noise attenuation. Systems and machine-readable media are also provided.

Noise mitigation using machine learning

Systems, methods, and devices are disclosed for detecting an active speaker in a two-way conference. Real time audio in one or more sub band domains are analyzed according to an echo cancellor model. Based on the analyzed real time audio, one or more audio metrics are determined from output from an acoustic echo cancellation linear filter. The one or more audio metrics are weighted based on a priority, and a speaker status is determined based on the weighted one or more audio metrics being analyzed according to an active speaker detection model. For an active speaker status, one or more residual echo or noise is removed from the real time audio based on the one or more audio metrics.

Methods and apparatus for transcoding digital video data are disclosed. In an embodiment, a transcoder () decodes a digital video block () using a first coding scheme, such as 8&#xd7;8 MPEG-2/4, to produce domain transformed data () and a motion vector (). The transcoder () then estimates an energy level of each sub-block in the digital video block () in the frequency domain (as opposed to the spatial domain), thereby reducing or eliminating the need for motion compensation. For each sub-block with an estimated energy level below a desired threshold (e.g., likely an all-zero sub-block), the transcoder () transcodes the sub-block by converting the motion vector () from the first coding scheme (e.g., MPEG-2/4) to the second coding scheme (e.g., H.264) (e.g., convert 8&#xd7;8 MPEG-2/4 vector to 4&#xd7;4 H.264 vector or reuse the MPEG-2/4 vector if all four sub-blocks are AZB and coding in H.264 as an 8&#xd7;8 block). The transcoded sub-block may then be used (e.g., stored or transmitted).

Methods and apparatus for transcoding digital video

A method is disclosed for performing a discrete cosine transform (DCT) using a microprocessor having an instruction set that includes SIMD floating point instructions. In one embodiment, the method includes: (1) receiving a block of integer data having C columns and R rows; and (2) for each row, (a) loading the row data into registers; (b) converting the row data into floating point form so that the registers each hold two floating point row data values; and (c) using SIMD floating point instructions to perform weighted-rotation operations on the values in the registers. Suitable SIMD floating point instructions include the pswap, pfmul, and pfpnacc instructions. For the row-DCT, the data values are preferably ordered in the registers so as to permit the use of these instructions. For the column-DCT, two columns are preferably processed in parallel using SIMD instructions to improve computational efficiency. An intermediate buffer may be used to avoid unnecessary conversions between integer and floating point format.

Two-dimensional discrete cosine transform using SIMD instructions

A method is disclosed for performing an inverse discrete cosine transform (IDCT) using a microprocessor having an instruction set that includes SIMD floating-point instructions. In one embodiment, the method includes: (1) receiving a block of integer data having C columns and R rows; and (2) for each row, (a) loading the row data into registers; (b) converting the row data into floating-point form so that the registers each hold two floating-point row data values; and (c) using SIMD floating-point instructions to perform weighted-rotation operations on the values in the registers. Suitable SIMD floating-point instructions include the pswap, pfmul, and pfpnacc instructions. For the row-IDCT, the data values are preferably ordered in the registers so as to permit the use of these instructions. For the column-IDCT, two columns are preferably processed concurrently using SIMD instructions to improve computational efficiency. An intermediate buffer may be used to avoid unnecessary conversions between integer and floating-point format.

Two-dimensional inverse discrete cosine transform using SIMD instructions

An efficient finite length POW10 calculation for MPEG audio encoding. A method for encoding an audio input signal includes storing a plurality of predetermined tonal values corresponding to a plurality of predetermined power levels. The method also includes receiving a plurality of input values each representative of a power level of a spectral component of the audio input signal at a corresponding frequency sub-band and accessing at least one corresponding tonal value of the plurality of predetermined tonal values. The method further includes generating an encoded output signal representative of the audio input signal by using at least one corresponding tonal value for each of the plurality of input values. Further, the storing of the plurality of predetermined tonal values is performed prior to the receiving of the plurality of input values.

Efficient finite length POW10 calculation for MPEG audio encoding

In a multimedia compression system such as is used in compressing data in a video stream, 'zero blocks' from an MPEG block array are identified prior to encoding and subjecting blocks to the process of quantized discrete cosine transform (DCT). A zero block as understood herein is an MPEG video data block that results when an unencoded block of video data is identical to an adjacent un-encoded block of video data, i.e., when there is little or no change between consecutive frames of video. The efficiency of the compression system is increased in this invention by not having to perform the functions of DCT, quantization, dequantization or inverse DCT on the identified zero blocks, but simply directly encoding zero blocks.

Zero-block encoding

The invention, in a first aspect, is a method for mitigating edge effects in a decompressed video image. The method comprises first reads an N&times;N group of pixels defining a vertical edge between two blocks in a video frame row by row into N registers, wherein N is a predetermined number defining the length of a filter. The content of the N registers is then transposed and then filtered in the filter. The filtered content of the N registers is then transposed and stored back from where it was read. In other aspects, the invention is a program storage device encoded with instructions that, when executed by a computer, perform such a method; a computer programmed to perform such a method; and a computing system capable of performing such a method.

Parallel edge filters in video codec

An improved form of half pixel accuracy motion estimation/compensation using the MPEG recommended half pixel approach resides in method and system. In the inventive method, an interpolated reference image is created before coding so that it can be preloaded into a cache memory and used whenever needed, without having to create it each time. To avoid redundant processing during accessing of subsampled interpolated data, the interpolated image using half pixel method is partitioned into four areas. The four areas are defined based on where in a 2&times;2 square region the pixels fall.

Efficient implementation of half-pixel motion prediction

A method of quantizing multiple input values in parallel using SIMD instructions is disclosed. In one embodiment, the method comprises (1) receiving a set of input values Xi; (2) operating on the set of input values to produce a set of binary mask values Ki that are related to the input values Xi by Ki&equals;&minus;1 if Xi>0, and Ki&equals;0 if Xi<0; (3) adding half of a quantization step Q to the input values Xi to obtain sum values; (4) subtracting half of a quantization step Q to the input values Xi to obtain difference values; (5) using the binary mask values to screen out sum values calculated from negative input values; (6) using a complementary binary mask to screen out difference values calculated from positive input values; and (7) combining the screened sum and difference values to determine prequantization values Yi that can be expressed Yi&equals;Xi&plus;Q/2 if Xi>0, and Yi&equals;Xi&minus;Q/2 if Xi&lE;0. The prequantization values may then be converted to floating point format, divided by Q, and restored to integer format to obtain the desired quantization values. Each of these operations can be performed in parallel on each of the values in the set using SIMD instructions.

Growing community of inventors

Austin, TX, United States of America

Wei-Lien Hsu