Sony Interactive Entertainment Inc.

Sony Corporation

Sony Electronics Inc

Sony Computer Entertainment Inc.

Hung-Ju Lee

Rathish Krishnan

Jason Naxin Wang

Eric Hsuming Chen

Deepali Arya

Ximin Zhang

Takefumi Nagumo

Jun Murayama

Chuen-Chien Lee

A technique to determine an optimum number of reserved bits for use in video encoding. The bits per coding unit is calculated as a function of bitrate, frame rate and frame size. Then for each coding unit, N reserved bits candidates are selected. For each test video the number of packets per frame is identified and the average number of packets per frame is calculated. The best reserved bits candidate out of the N reserved bits candidates is identified, and then a best fitted linear model between best reserved bits candidate and bits per coding unit is determined.

Reserved bit prediction for slice size control with pipelined video encoding

Video stream data is selectively scaled so that sections within regions of interest (ROI) maintain high resolution while areas not within the region of interest are down-scaled to reduce bandwidth cost of transmission. A low compression encoder compresses sections of a video frame corresponding to one or more ROI without motion search or prediction mode decision to generate low-compression section data. The video frame is downscaled and a high compression encoder compresses the resulting downscaled video frame with prediction mode decision to generate high-compression frame data.

Foveated video link for VR with gaze tracking

A derived quantization parameter for a section of a currently encoding picture of a plurality of pictures is incremented to produce an updated quantization parameter when the derived quantization parameter is less than a minimum quantization parameter for the currently encoding picture. The section is then encoded using the updated quantization parameter. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

Picture quality oriented rate control for low-latency streaming applications

A method, system, and computer readable medium for encoding one or more input digital frames with obscured negative regions of interest. The method comprising, comparing frame data to mask data for one or more sections of an input digital frame to determine whether or not a given section of the input digital frame is to be obscured, setting one or more encoding parameters to values that result in corresponding obscured sections when decoded after encoding for each section of the frame that is to be obscured, encoding the one or more sections of the input digital frame to generate corresponding encoded frame data and storing or transmitting the encoded frame data.

Negative region-of-interest video coding

Gaze tracking data is analyzed to determine one or more regions of interest within an image of a video stream. The video stream data is selectively scaled so that sections within the regions of interest maintain high resolution while areas not within the region of interest are down-scaled to reduce bandwidth cost of transmission. A scheme for reduction of motion sickness by reducing the size of the high resolution area is also claimed.

Foveated video link for VR, low latency wireless HMD video streaming with gaze tracking

In aspects of the present disclosure, a software encoder augments a hardware encoder by implementing portions of a video encoding task that are not supported by the hardware encoder while the hardware encoder implements other portions of the encoding task. The use of a software encoder to augment a hardware encoder in this manner can extend the useful life of a hardware encoder, allow the system to adapt to changes in video coding standards, and in some cases improve performance of encoding implemented by hardware or software alone. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

Software hardware hybrid video encoder

Digital pictures may be encoded and decoded in a way that optimizes parallelization. Selected sections of a digital picture in a first list are encoded independently of encoding any of their neighbor sections. Sections in a second list located to the right of corresponding sections in the first list are encoded using at most the section to their immediate left. Sections in a third list located to the right of a corresponding section in the second list and immediately below a corresponding section in the first list are encoded using at most the sections to the immediate left, immediately above and immediately to its above-right. All other sections of the picture may be encoded in a normal manner.

Parallel intra-prediction encoding/decoding process utilizing PIPCM and/or PIDC for selected sections

An advanced constant quality rate control algorithm with new features and enhancements. can be applicable to different kinds of video scenes, in addition to static or less motion scenes. Statistical data is calculated at the beginning of encoding for each line of sections of a frame. Based on the statistical data, which case of a plurality of cases to be executed is determined and maximum and minimum QP values are determined according to the determined case. Actual maximum and minimum QP of the current line using is maintained using the maximum and minimum QP and a final QP value is determined for all sections in each line. A lambda value for encoding is determined from the final QP. At the end of encoding the picture, the number of frames encoded using constant quality is counted, the maximum and minimum QP and VBV fullness for each line is updated.

Advanced picture quality oriented rate control for low-latency streaming applications

A random access point is generated in a stream of coded digital pictures containing a plurality of predictive coded frames that have two or more predictive coded frames in which one or more subsections of each of the two or more predictive coded frames are intra coded. Information is added to a stream of digital pictures that identifies for a decoder which of two or more predictive-coded frames in the stream have intra-coded subsections at different portions that can be combined to form a patch frame.

Random access point (RAP) formation using intra refreshing technique in video coding

A cost function for use in an encoding mode decision for encoding a section of a digital picture is determined using a non-default relationship between Lagrange multiplier (&#x3bb;) values and quantization parameter (QP) values to determine the cost function when a number of bits per section (NBS) of the digital picture is in a predetermined range and determined using a default relationship between the &#x3bb; values and the QP values to determine the cost function otherwise. The determined cost function is used to make the section encoding mode decision and the section is encoded according to the encoding mode decision. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

Adaptive picture section encoding mode decision control

Three-dimensional motion mapping is disclosed. A reverse three-dimensional transform for a set of anchor pixels in the current frame is performed using a model-view matrix, a projection matrix, and view parameters associated with the computer game. Reverse transformed anchor pixels are indicated as occluded if the estimated depth for the pixel in the previous frame is greater than a real depth for the pixel in the previous frame or otherwise indicated as visible. A motion vector map for a subset of the reverse transformed pixels is constructed. A best target motion vector for a first set of sections in the current frame is derived or motion estimation is performed for sections in a second set that contain occluded pixels. Target motion vectors for groups of two or more sections of the current frame comprising two or more adjoining sections each with best target motion vectors are derived.

Three-dimensional motion mapping for cloud gaming

Computer-implemented rate control in encoding one or more video pictures is disclosed. A quantization parameter (QP) to be used for encoding a current picture one of the one or more video pictures is determined using rate control data that takes picture type, picture complexity, and a target bitcount into account in determining the QP. The current picture is encoded into a compressed format using the QP to generate an encoded current picture. The rate control data is updated based on statistical data determined from the encoded current picture.

Picture-level rate control for video encoding a scene-change I picture

A video system is provided including applying a transform to a picture for generating a transformed block of the picture; determining a type of the picture; selecting a coding mode with the transformed block based on the type of the picture; and applying the coding mode to the picture for encoding and displaying the picture.

Growing community of inventors

Pleasanton, CA, United States of America

Hung-Ju Lee