Sony Interactive Entertainment Inc.

Sony Interactive Entertainment America LLC

Sony Computer Entertainment America LLC

Sony Computer Entertainment Inc.

Advanced Micro Devices Corporation

Mark Evan Cerny

David Simpson

Kelvin M Yong

Tobias Berghoff

Florian A Strauss

Jason Scanlin

Roelof Roderick Colenbrander

Pal-Kristian Engstad

Simon Whittaker

John Paul Bates

Akiyuki Hatakeyama

Richard B Stenson

Jeffrey Litz

Varun Bhadauria

Simon Pilgrim

Boris Batkin

Jonathan J Olick

Michael Mantor

Steven Osman

Payton R White

Keisuke Inoue

Howard Berkey

Jun Murakawa

Motoi Kaneko

Yoshinori Washizu

Christopher Ho

Dmitri Shtilman

Eric Langyel

Nathaniel Hoffman

Jon Olick

Atilla Vass

John Morgan

A method of cloud gaming is disclosed. The method including receiving an encoded video frame at a client, wherein a server executes an application to generate a rendered video frame which is then encoded at an encoder at the server as the encoded video frame, wherein the encoded video frame includes one or more encoded slices that are compressed. The method including decoding the one or more encoded slices at a decoder of the client to generate one or more decoded slices. The method including rendering the one or more decoded slices for display at the client. The method including begin displaying the one or more decoded slices that are rendered before fully receiving the one or more encoded slices at the client.

Overlapping rendering, streamout, and display at a client of rendered slices of a video frame

Methods for graphics processing are provided. One example method includes executing a plurality of kernels using a plurality of graphics processing units (GPUs), wherein responsibility for executing a corresponding kernel is divided into one or more portions each of which being assigned to a corresponding GPU. The method includes generating a plurality of dependency data at a first kernel as each of a first plurality of portions of the first kernel completes processing. The method includes checking dependency data from one or more portions of the first kernel prior to execution of a portion of a second kernel. The method includes delaying execution of the portion of the second kernel as long as the corresponding dependency data of the first kernel has not been met.

Accessing local memory of a GPU executing a first kernel when executing a second kernel of another GPU

A method for graphics processing. The method including rendering graphics for an application using graphics processing units (GPUs). The method including using the plurality of GPUs in collaboration to render an image frame including a plurality of pieces of geometry. The method including during the rendering of the image frame, subdividing one or more of the plurality of pieces of geometry into smaller pieces, and dividing the responsibility for rendering these smaller portions of geometry among the plurality of GPUs, wherein each of the smaller portions of geometry is processed by a corresponding GPU. The method including for those pieces of geometry that are not subdivided, dividing the responsibility for rendering the pieces of geometry among the plurality of GPUs, wherein each of these pieces of geometry is processed by a corresponding GPU.

Reassigning geometry based on timing analysis when rendering an image frame

A method is disclosed including setting, at a server, a server VSYNC signal to a server VSYNC frequency. The server VSYNC signal corresponding to generation of video frames during frame periods for the server VSYNC frequency. The method including setting, at a client, a client VSYNC signal to a client VSYNC frequency. The method including sending compressed video frames from the server to the client over a network using the server VSYNC signal, wherein the compressed video frames are based on the generated video frames. The method including decoding and displaying, at the client, the compressed video frames. The method including analyzing the timing of one or more client operations to adjust the relative timing between the server VSYNC signal and the client VSYNC signal, as the client receives the compressed video frames.

Synchronizing a frequency of a client VSYNC signal to a frequency of a server VSYNC signal

A method for cloud gaming. The method including generating a video frame when executing a video game at a server. The method including performing a scan-out process by scanning the video frame scanline-by-scanline and one or more user interface features scanline-by-scanline to one or more input frame buffers, and compositing and blending the video frame and the one or more user interface features into a modified video frame. The method including scanning in the scan-out process the modified video frame to an encoder at the server scanline-by-scanline. The method including beginning in the scan-out process scanning the video frame and the one or more user interface features to the one or more input frame buffers at a corresponding flip-time for the video frame.

Beginning scan-out process at flip-time for cloud gaming applications

A method for encoding including executing game logic built on a game engine of a video game at a cloud gaming server to generate video frames. The method including executing scene change logic to predict a scene change in the video frames based on game state collected during execution of the game logic. The method including identifying a range of video frames that is predicted to include the scene change. The method including generating a scene change hint using the scene change logic, wherein the scene change hint identifies the range of video frames, wherein the range of video frames includes a first video frame. The method including delivering the first video frame to an encoder. The method including sending the scene change hint from the scene change logic to the encoder. The method including encoding the first video frame as an I-frame based on the scene change hint.

Encoding a video frame as a reference frame based on a scene change hint at a cloud gaming server

A method, system and computer readable medium for running a legacy application on a non-legacy device. Operating parameters of the non-legacy device when running the legacy application are set based on one or more pre-determined heuristics for adjustment of operating parameters of the newer system when running the legacy application on the non-legacy device from one or more performance metrics and other performance information.

Running a legacy application on a non-legacy device with application-specific operating parameters for backwards compatibility

A method for executing a game by a computing system that uses a central processing unit (CPU) and graphics processing unit (GPU) for generating video frames. A draw call is generated for a video frame by the CPU. At bind time, i.e. writing of the GPU commands by the CPU using a GPU API, asset aware data (AAD) is written to the command buffer, and loading of one or more level of detail (LOD) data from an asset store to system memory is requested. The GPU executes the draw call for the frame using LOD data written to the system memory, the GPU using at least a minimum of LOD data based on the AAD. Additionally, the GPU uses information regarding the LOD load state when executing the draw call, in order to avoid access to LODs not yet loaded.

Asset aware data for using minimum level of detail for assets for image frame generation

A method for cloud gaming. The method including receiving one or more encoded slices of a video frame at a client, wherein the video frame was generated at a server while executing a video game, and encoded by an encoder at the server into the one or more encoded slices. The method including decoding a first encoded slice at a decoder of the client before fully receiving the one or more encoded slices of the video frame.

Overlapping operations at a server and a client for a video frame to reduce latency in cloud gaming applications

A method including rendering graphics for an application using graphics processing units (GPUs). The method including dividing responsibility for processing pieces of geometry of an image frame during an analysis pre-pass phase of rendering between the plurality of GPUs, wherein each of the pieces of geometry is assigned to a corresponding GPU. The method including determining in the analysis pre-pass phase overlap of each of the pieces of geometry with each screen regions. The method including generating information at the GPUs regarding the pieces of geometry and their relations to the screen regions based on the overlap of each of the pieces of geometry with each of the screen regions. The method including assigning the screen regions to the GPUs based on the information for purposes of rendering the pieces of geometry during a subsequent phase of rendering.

Performing geometry analysis before multi-GPU rendering of geometry

A method, system and non-transitory computer readable instructions for application patching comprising, concatenating compressed data or uncompressed data or a mixture of compressed and uncompressed data into a continuous data set into a continuous data set and dividing the continuous data set into variable sized data chunks. Compressing each of the variable sized data chunks and dividing each of the variable sized data chunks into fixed size data blocks. Encrypting the fixed size data blocks to generate encrypted fixed size data blocks and storing the encrypted fixed sized data blocks or sending the encrypted fixed size data blocks over a network.

Application patching using variable-sized units

A method is disclosed including setting, at a plurality of devices, a plurality of VSYNC signals to a plurality of VSYNC frequencies, wherein a corresponding device VSYNC signal of a corresponding device is set to a corresponding device VSYNC frequency. The method including sending a plurality of signals between the plurality of devices, which are analyzed and used to adjust the relative timing between corresponding device VSYNC signals of at least two devices.

Synchronizing display of video frames between client devices in a multi-tenant configuration

A method is disclosed including setting, at a server, a server VSYNC signal to a server VSYNC frequency defining a plurality of frame periods. The server VSYNC signal corresponding to generation of a plurality of video frames at the server during the plurality of frame periods. The method including setting, at a client, a client VSYNC signal to a client VSYNC frequency. The method including sending a plurality of compressed video frames based on the plurality of video frames from the server to the client over a network using the server VSYNC signal. The method including decoding and displaying, at the client, the plurality of compressed video frames. The method including analyzing the timing of one or more client operations to set the amount of frame buffering used by the client, as the client receives the plurality of compressed video frames.

Dynamic client buffering using server and client VSYNC signals of received video frames for cloud gaming

A method for cloud gaming. The method including generating a plurality of video frames when executing a video game at a cloud gaming server. The method including encoding the plurality of video frames at an encoder bit rate, wherein the plurality of video frames that is compressed is transmitted to a client from a streamer of the cloud gaming server. The method including measuring a maximum receive bandwidth of a client. The method including monitoring the encoding of the plurality of video frames at the streamer. The method including dynamically tuning a parameter of the encoder based on the monitoring of the encoding.

Buffer drain rate tuning to a measured maximum receive bandwidth measured for a client device when streaming

A method for graphics processing. The method including rendering graphics for an application using a plurality of graphics processing units (GPUs). The method including using the plurality of GPUs in collaboration to render an image frame including a plurality of pieces of geometry. The method including during a pre-pass phase of rendering, generating information at the GPUs regarding the plurality of pieces of geometry and their relation to a plurality of screen regions. The method including assigning the plurality of screen regions to the plurality of GPUs based on the information for purposes of rendering the plurality of pieces of geometry in a subsequent phase of rendering.

Growing community of inventors

Los Angeles, CA, United States of America

Mark Evan Cerny