Vidyo, Inc.

Google Inc.

Danny Hong

Jill MacDonald Boyce

Alexandros Eleftheriadis

Ofer Shapiro

Thomas Wiegand

Stephan Wenger

Wonkap Jang

Ramachandra Tahasildar

Jonathan Michael Lennox

Stephen Cipolli

Mehmet Reha Civanlar

Alex Sukhanov

Reha Civanlar

In Suk Chong

Richard Xie

Jill Boyce

Beril Erkin

Zhuo Chen

Won Kap Jang

Jonathan Lennox

Yinqing Zhao

Jang Wonkap

Pre-encoding noise parameterization techniques mitigate or eliminate banding and other graphical artifacts in video frames for decoding and presentation by a client device. For one or more input video frames, a quantization parameter associated with the input video frames is identified. Noise synthesis parameters are determined based on the identified quantization parameter, and the input video frames are encoded for transmission. The encoded video frames are transmitted to the client device along with the determined noise synthesis parameters, for use by the client device in generating synthetic noise to add to resulting video frames decoded by the client device.

Parameterized noise synthesis for graphical artifact removal

Techniques for encoding game content for real-time cloud-gaming, and more specifically to using an adaptive statistical model in a moving window of previous frames up to a current frame to adaptively compute a lower-bound of a quantization parameter (QP) that corresponds to a chosen peak signal to noise ratio (PSNR) target. The result of this PSNR-guided encoding is significantly reducing the size of encoded frames (i.e., undershooting a target bit rate when there are no subjective quality losses on those frames using their PSNR as the guidance). These reductions mitigate bandwidth issues and improve the overall user experience because less bits transferred results in faster network transmission and decoding, as well as less platform overhead.

Content-adaptive signal-to noise-ratio-guided two-pass video encoding for cloud gaming

A cloud-based extended reality (XR) system includes a server configured to encode a set of frames each associated with an XR scene to be displayed. To encode the set of frames, the server estimates a total number of encoded output bits for the set of frames based on a set of quantization parameters (QPs). The set of QPs includes a corresponding QP for each frame of the set of frames and one or more predetermined relationships between the corresponding QPs. The server then compares the estimated total number of encoded output bits to a target frame size threshold. Based on the estimated total number of encoded bits being outside the target frame size threshold, the server updates the set of QPs so as to maintain the predetermined relationships between the QPs.

Multi-channel video rate control for extended reality streaming

A two-pass encoding operation is implemented to encode one or more gaming frames into a game stream. The two-pass encoding operation includes a first encoding pass performed on a current frame. As a result of the first encoding pass, an estimated complexity for the current frame is determined. The resulting estimated complexity is then modulated according to a quality difference between reference frames used during the first pass encoding and a subsequent second pass encoding. Based on the modulated complexity, a quantization parameter is determined for the current frame that is then used to perform a second pass encoding on the current frame, resulting in an encoded frame. This encoded frame is then transmitted as part of a stream to a client system.

Low-delay two-pass frame-level rate control using an adjusted complexity

Disclosed are techniques for temporal scalable video coding. By including a temporal_switching_point_flag and/or a temporal_nesting_flag into a normative part of the scalable bitstream, reference picture management in decoder can be simplified, and media-aware network elements and decoder can easily identify those pictures from which onward a higher temporal layer can be added.

High layer syntax for temporal scalability

Disclosed are techniques for creating, coding, decoding, and using, rotation information related to one or more coded pictures in non-normative parts of a coded video bitstream.

Render-orientation information in video bitstream

Techniques to determine a level id in a layered bitstream are disclosed, such that the same level id can be used for all layers of a scalable bitstream. Also disclosed are techniques to signal level ids for sub bitstreams that can be indicative of lower computational requirements than the level id used for the layered bitstream.

Level signaling for layered video coding

Scalable video codecs are provided for use in videoconferencing systems and applications hosted on heterogeneous endpoints/receivers and network environments. The scalable video codecs provide a coded representation of a source video signal at multiple temporal, quality, and spatial resolutions.

System and method for scalable and low-delay videoconferencing using scalable video coding

The disclosed subject matter describes a new pattern description that can be part of an SEI message or a parameter set, and can be used to describe a temporal picture coding structure and associated bitrates and frame rates. The knowledge of the coding structure can benefit transraters, bitstream extractors, and digital video recorders. Decoders can utilize the coding structure information for example, to decode pictures in parallel, using multiple threads/cores.

Techniques for describing temporal coding structure

Disclosed are techniques for encoding and decoding layered video where the non-temporal enhancement layers and their respective temporal enhancement layers, comply with a scalable video coding standard or technology, and the base layer and its respective temporal enhancement layers does not comply with the same scalable video coding standard or technology. A Video Parameter Set that comprises information about the relationship of layers includes a syntax element indicative of the derivation mechanism for a temporal layer associated with a NAL unit coded in a first coding technology, for example HEVC. For one value of the syntax element, the derivation mechanism is to set the temporal layer of the base layer NAL unit to the value coded in the header of the encapsulating NAL unit, which can be an HEVC NAL unit. For another value, the derivation mechanism is to imply the value of temporal base layer for the first NAL unit.

Hybrid video coding techniques

Systems and methods for error resilient transmission, rate control, and random access in video communication systems that use sealable video coding are provided. Error resilience is obtained by using information from low resolution layers to conceal or compensate loss of high resolution layer information. The same mechanism is used for rate control by selectively eliminating high resolution layer information from transmitted signals, which elimination can be compensated at the receiver using information from low resolution layers. Further, random access or switching between low and high resolutions is also achieved by using information from low resolution layers to compensate for high resolution spatial layer packets that may have not been received prior to the switching time.

System and method for providing error resilience, random access and rate control in scalable video communications

Systems and methods for error resilient transmission, rate control, and random access in video communication systems that use scalable video coding are provided. Error resilience is obtained by using information from low resolution layers to conceal or compensate loss of high resolution layer information. The same mechanism is used for rate control by selectively eliminating high resolution layer information from transmitted signals, which elimination can be compensated at the receiver using information from low resolution layers. Further, random access or switching between low and high resolutions is also achieved by using information from low resolution layers to compensate for high resolution spatial layer packets that may have not been received prior to the switching time.

Techniques for video decoding include cross-layer prediction from a base layer to an SNR or spatial enhancement layer of spatial intra prediction mode information. In one example embodiment, the Most Probable Mode (MPM) for intra prediction of an enhancement layer block is predicted from the spatially corresponding base layer block.

Growing community of inventors

New York, NY, United States of America

Danny Hong