Broadcom Corporation

Agora Lab, Inc.

Avago Technologies General IP (singapore) Pte. Ltd.

Continental Teves, Ag and Company Ohg

Sheng Zhong

Alexander Garland MacInnis

Vivian T Hsiun

Xiaodong Xie

José Roberto Alvarez

Jose′ R Alvarez

Wei Dai

Zesen Zhuang

Darren Duane Neuman

Brian Francis Schoner

David Chaohua Wu

Yue Feng

Jose' R Alvarez

Shifu Zhou

Bin Zhao

Alexander G Macinnis

Kimming So

Chenhui Feng

Ramkumar Prakasam

Zhonghou Zheng

Alexander G Maclnnis

Xiaodong Xic

Jiajie Jian

Tao Duan

Xiajun Gu

Jose′ R I Alvarez

Mingyue Hao

A method for allocating bitrate in video communications is disclosed. The method includes encoding, via an encoder, a video stream that is partitioned into tiles associated with a viewing angle of the video stream, whereby encoding the video stream includes allocating each of the tiles a bitrate. The method also includes determining an image quality score associated with rendering of the tiles at a first iteration, whereby rendering of the tiles includes decoding and displaying the tiles at the first iteration in the viewing angle of the video stream with the bitrate. The method further includes, responsive to determining the image quality score, modifying the bitrate allocated to at least some of the tiles such that rendering of the tiles at a second iteration that is after the first iteration includes decoding and displaying the tiles with the modified bitrate.

Tile encoding bitrate allocation for low-latency video communication systems

A method includes maintaining a phase-out list that includes tiles of a current viewport of a wide-angle image that are predicted to no longer be in the viewport after a time duration d from a current time t. Tiles predicted to be viewed at a time that is later than the current time t are obtained. Those of the predicted tiles that are not in a receiver list are added to the receiver list. Tiles that form a current viewing area are rendered using the receiver list. Tiles of the receiver list and a phase-out list are associated with respective display times. Rendering the tiles includes decoding and displaying tiles of the receiver list having display times that are not larger than the current time t and stopping decoding and displaying tiles of the phase-out list associated with the display times that are not larger than the current time t.

Real-time wide-angle video communication system

A method for image generation based on a Generative AI Network. The Generative AI Network includes a generator and an encoder. The method includes determining, by the encoder, a first encoding E(Y) of a target image Y; generating, by the generator, a generated image G(Z) corresponding to the target image Y, wherein the generated image G(Z) is located in a close vicinity of a target neighborhood of the target image Y, and outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the Generative AI Network; and generating, by the encoder, a second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map the target image Y and the generated image G(Z) to the latent space.

Optimizing generative networks via latent space regularizations

A method, an apparatus, and a system for video communications include: transmitting, from a first apparatus using a network, a first video stream of a first user to a second apparatus of a second user, wherein the first user is in video communication with the second user; receiving, from the second apparatus using the network, a second video stream of the second user; determining, by a processor, a reaction of the second user to an area of interest in the first video stream using the second video stream; and updating, in response to the reaction of the second user to the area of interest in the first video stream, a parameter for encoding the area of interest in the first video stream at the first apparatus.

Video tagging for video communications

A system and method for determining the performance of a real-time communication application, such as a video codec and transmission of the coded data. The method factors in burst data packet loss in determining a performance measure. The method uses a Hidden Markov Model on a set of data packets of a unit of data to determine the performance measure in the form a probability that all the data packets within the set are received. The measure is used to fine tune the system settings. The method causes the real-time communication application to adjust parameters of the application for improved transmission of data packets. The adjustment of a parameter can be an increase or a decrease.

System and method for determining network transmission performance in real-time communication over internet

A system and method for determining the performance of a video codec for real-time communication applications. A set of network conditions is used to simulate certain typical real-world network conditions. A coded video stream is transmitted from the transmitting end to the receiving end under these conditions. The end-to-end latency and received video fluency are measured along with a set of existing video quality measures by a video codec performance evaluation system including a network model and a video codec quality analyzer. The decodable frame ratio, latency and video fluency are used as the performance metrics for real-time communication video quality evaluation. The video codec performance evaluation system does not send any video data over a network when it determines the real-time communication quality of the video codec.

System and method for determining video codec performance in real-time communication over internet

A method for image generation based on a Generative Adversarial Network (GAN) including a generator, a discriminator, and an encoder, wherein outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the GAN, the method including generating, by the encoder, a first encoding E(Y) of a target image Y and a second encoding E(G(Z)) of a generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map Y and G(Z) to the latent space having a lower dimension than dimensionality of Y and G(Z), wherein the encoder is trained to minimize the differences between the first and second encodings E(Y) and E(G(Z)), and the generator is trained by using the first and second encodings E(Y) and E(G(Z)) as part of a loss function.

Optimizing supervised generative adversarial networks via latent space regularizations

Respective bandwidth information of respective connected devices of active media sessions are obtained. At least one additional media session is detected to have become active. A first total bandwidth of the at least one additional media session is determined. At least one scaling factor is determined using the respective bandwidth information and the first total bandwidth. The at least one scaling factor is transmitted to at least one connected device of the at least one of the respective connected devices. The at least one scaling factor is determined so as to keep a second total bandwidth of the active media sessions and the at least one additional media session below or equal an upper bandwidth limit of the communications network.

Video communications network with value optimization

A method of training a generator G of a Generative Adversarial Network (GAN) includes receiving, by an encoder E, a target data Y; receiving, by the encoder E, an output G(Z) of the generator G, where the generator G generates the output G(Z) in response to receiving a random sample Z and where a discriminator D of the GAN is trained to distinguish which of the G(Z) and the target data Y; training the encoder E to minimize a difference between a first latent space representation E(G(Z)) of the output G(Z) and a second latent space representation E(Y) of the target data Y, where the output G(Z) and the target data Y are input to the encoder E; and using the first latent space representation E(G(Z)) and the second latent space representation E(Y) to constrain the training of the generator G.

Training a generator G of a GAN includes generating, by G and in response to receiving a first input Z, a first output G(Z); generating, by an encoder E of the GAN and in response to receiving the first output G(Z) as input, a second output E(G(Z)); generating, by G and in response to receiving the second output E(G(Z)) as input, a third output G(E(G(Z))); generating, by E and in response to receiving the third output G(E(G(Z))) as input, a fourth output E(G(E(G(Z)))); training E to minimize a difference between the second output E(G(Z)) and the fourth output E(G(E(G(Z)))); and using the second output E(G(Z)) and fourth output E(G(E(G(Z)))) to constrain a training of the generator G. G(Z) is an ambient space representation Z. E(G(Z)) is a latent space representation of G(Z). G(E(G(Z))) is an ambient space representation of E(G(Z)). E(G(E(G(Z)))) is a latent space representation of G(E(G(Z))).

Optimizing unsupervised generative adversarial networks via latent space regularizations

A method for content moderation in real-time communication includes receiving a first portion of a media stream from a sending user; scaling the first portion of the media stream to obtain a scaled first portion of the media stream; transmitting, to a centralized content moderation device, the scaled first portion of the media stream; receiving, from the centralized content moderation device, a centralized determination of whether the first portion is appropriate; and, in response to the centralized determination indicating that the first portion of the media stream is inappropriate: blurring the first portion to obtain a blurred first portion; and transmitting, to a receiving user, the blurred first portion.

Scalable multi-level collaborative content moderation

A method for use in a communications network includes obtaining, at a control node, respective bandwidth information of respective connected devices of media sessions; determining at least one scaling factor of a respective bandwidth of at least one of the respective connected devices of a media session of the media sessions; and transmitting the at least one scaling factor to the at least one of the respective connected devices. The at least one of the respective connected devices uses the scaling factor to adjust an available bandwidth. The at least one of the respective connected devices uses the scaling factor to adjust encoding parameters of the media session.

System and method for decoding digital video data. The decoding system employs hardware accelerators that assist a core processor in performing selected decoding tasks. The hardware accelerators are configurable to support a plurality of existing and future encoding/decoding formats. The accelerators are configurable to support substantially any existing or future encoding/decoding formats that fall into the general class of DCT-based, entropy decoded, block-motion-compensated compression algorithms. The hardware accelerators illustratively comprise a programmable entropy decoder, an inverse quantization module, a inverse discrete cosine transform module, a pixel filter, a motion compensation module and a de-blocking filter. The hardware accelerators function in a decoding pipeline wherein at any given stage in the pipeline, while a given function is being performed on a given macroblock, the next macroblock in the data stream is being worked on by the previous function in the pipeline.

Video decoding system supporting multiple standards

A method of training a generator G of a Generative Adversarial Network (GAN) includes generating a real contextual data set {x, . . . , x} for a high resolution image Y; generating a generated contextual data set {g, . . . , g} for a generated high resolution image G(Z); calculating a perceptual loss Lvalue using the real contextual data set {x, . . . , x} and the generated contextual data set {g, . . . , g}; and training the generator G using the perceptual loss Lvalue. The generated high resolution image G(Z) is generated by the generator G of the GAN in response to receiving an input Z, where the input Z is a random sample that corresponds to the high resolution image Y.

Effective structure keeping for generative adversarial networks for single image super resolution

A method of training a generator G of a Generative Adversarial Network (GAN) includes receiving, by an encoder E, a target data Y; receiving, by the encoder E, an output G(Z) of the generator G, where the generator G generates the output G(Z) in response to receiving a random sample Z that is a noisy sample, and where a discriminator D of the GAN is trained to distinguish which of the G(Z) and the target data Y is real data; training the encoder E to minimize a difference between a first latent space representation E(G(Z)) of the output G(Z) and a second latent space representation E(Y) of the target data Y, where the output G(Z) and the target data Y are input to the encoder E; and using the first latent space representation E(G(Z)) and the second latent space representation E(Y) to constrain the training of the generator G.

Growing community of inventors

Fremont, CA, United States of America

Sheng Zhong