Sony Interactive Entertainment Inc.

Sony Interactive Entertainment Europe Limited

Rolls-Royce Corporation

Andrew Swann

Hugh Alexander Dinsdale Spencer

Nicholas Anthony Edward Ryan

Simon Andrew St John Brislin

Pritpal Singh Panesar

Owen Lyon

Hugh Alexander Dinsdale Spencer Spencer

A method of video playback includes obtaining a 3D reconstruction of at least part of an environment, obtaining video footage of an activity within that at least part of the environment, obtaining data indicating a position and orientation of a first camera within the at least part of the environment that recorded the obtained video footage, and rendering a view of the video footage in alignment with a corresponding view of at least part of the 3D reconstruction, responsive to the position and orientation of the first camera within the at least part of the environment.

Apparatus and method of video playback

A method of generating a representation of a virtual environment includes obtaining a mesh of at least part of the virtual environment; detecting the continuous and non-overlapping portions of that mesh, hereafter 'patches'; populating a texture map with representations of the patches by, for successive patches in order of large to small, identifying a position on the texture map where a current patch meets a non-overwrite criterion, wherein the a non-overwrite criterion requires that texels corresponding to the current patch do not overwrite texels of any previously positioned patch, and successive positions for each current patch are tested at intervals starting from a common origin position; and associating the current patch with the identified position; setting colour values for the texels of the texture map responsive to colour information corresponding to the obtained mesh; and outputting the obtained mesh and texture map for use by a third party.

Apparatus and method of generating a representation of a virtual environment

A method of mapping a virtual environment comprises obtaining a first sequence of video images output by a videogame title; obtaining a corresponding sequence of in-game virtual camera positions at which the video images were created; obtaining a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images; and for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, obtain mapping points corresponding to a sampling distribution of points over the area of a respective video image and their associated depth values; wherein respective mapping points are obtained by projecting co-ordinated derived from the sample points from the video image and associated depth values back into a 3D game world co-ordinate system of the videogame title; thereby obtaining a point cloud dataset of mapping points corresponding to the first sequence of video images.

Apparatus and method of mapping a virtual environment

A video recording method includes recording a sequence of video images output by a videogame, recording a sequence of depth buffer values for a depth buffer used by the videogame, recording a sequence of in-game virtual camera positions used to generate the video images, recording one or more in-game events and their respective in-game positions, associating the depth buffer value sequence, the in-game virtual camera position sequence, and an identifier for the videogame, with the video image sequence, and associating the one or more in-game events and their respective in-game positions with the identifier for the videogame. A video playback method includes obtaining a video recording of a videogame playing session, comprising a sequence of video images, obtaining a depth buffer value sequence, an in-game virtual camera position sequence, and an identifier for the videogame that are associated with the video recording, obtaining data indicative of a statistically significant in-game event and an in-game event position, calculating a position within the current video image corresponding to the in-game event position, in dependence upon the associated in-game virtual camera position and depth buffer values, and augmenting the current video image with a graphical representation of the statistically significant in-game event, responsive to the calculated position.

Video recording and playback systems and methods

A method of detecting significant footage for recording from a videogame includes obtaining position information for a target object within a virtual environment of the videogame, obtaining depth buffer information for a current position of a virtual camera used to generate a current image of the virtual environment for display by the videogame, calculating a first distance along a line between the current position of the virtual camera and the obtained position of the target object, detecting whether a depth buffer value along the line corresponds to at least a threshold distance from the virtual camera, the threshold distance being based upon the calculated first distance, and if so, associating the current image with an indicator that the image is significant for the purposes of recording footage from the videogame.

Methods and apparatus provide for: recording a first sequence of video images output by a videogame, recording a sequence of depth buffer values for a depth buffer used by the videogame, recording a sequence of in-game virtual camera positions used to generate the video images, recording one or more in-game events and their respective in-game positions, generating a second sequence of video images encoding the depth buffer value sequence, and associating the in-game virtual camera position sequence, with at least one of the first and second sequence of video images. Methods and apparatus also provide for: obtaining a first video recording of a videogame playing session comprising a sequence of video images, obtaining a second video recording generated by encoding a depth buffer value sequence, obtaining data indicative of an in-game virtual camera position sequence that is associated with at least one of the obtained video recordings, obtaining data indicative of a statistically significant in-game event and an in-game event position, calculating a position within a current video image of the first video recording corresponding to the in-game event position, in dependence upon the associated in-game virtual camera position and decoded depth buffer values, and augmenting the current video image with a graphical representation of the statistically significant in-game event, responsive to the calculated position.

A method of mapping a virtual environment includes: obtaining a first sequence of video images output by a videogame title; obtaining a corresponding sequence of in-game virtual camera positions at which the video images were created; obtaining a corresponding sequence of depth buffer values for a depth buffer used by the videogame whilst creating the video images; and, for each of a plurality of video images and corresponding depth buffer values of the obtained sequences, obtain mapping points corresponding to a selected predetermined set of depth values corresponding to a predetermined set of positions within a respective video image; where for each pair of depth values and video image positions, a mapping point has a distance from the virtual camera position based upon the depth value, and a position based upon the relative positions of the virtual camera and the respective video image position, thereby obtaining a map dataset of mapping points corresponding to the first sequence of video images.

Methods and apparatus provide for: recording a sequence of video images output by a videogame, recording a sequence of depth buffer values for a depth buffer used by the videogame, recording a sequence of in-game virtual camera positions used to generate the video images, recording one or more in-game events and their respective in-game positions, associating the depth buffer value sequence, the in-game virtual camera position sequence, and an identifier for the videogame, with the video image sequence, and associating the one or more in-game events and their respective in-game positions with the identifier for the videogame. Methods an apparatus also provide for: obtaining a video recording of a videogame playing session, comprising a sequence of video images, obtaining a depth buffer value sequence, an in-game virtual camera position sequence, and an identifier for the videogame that are associated with the video recording, obtaining data indicative of a statistically significant in-game event and an in-game event position, calculating a position within the current video image corresponding to the in-game event position, in dependence upon the associated in-game virtual camera position and depth buffer values, and augmenting the current video image with a graphical representation of the statistically significant in-game event, responsive to the calculated position.

Growing community of inventors

London, United Kingdom

Andrew Swann