Sony Computer Entertainment Europe Limited

Sony Interactive Entertainment Europe Limited

Sony Interactive Entertainment Inc.

Ian Henry Bickerstaff

Simon Mark Benson

Sharwin Winesh Raghoebardajal

Tomohiro Oto

Sharwin Winesh Raghoebardayal

Andrew Damian Hosfield

Nicola Orrù

Oliver Mark Wright

Hidehiko Morisada

William John Dudley

Ian Michael Hocking

Darren Myatt

Jeremy David Ashforth

Bruno Ribeiro

Nigel Kershaw

Nico Orru

Oliver Wright

Methods and apparatus provide for generating an image by way of acquiring information relating to at least one of a position and a rotation of a camera. The image is generated for display on a display unit. A rate at which the image is generated is at a first frequency, which is lower than a second frequency corresponding to a frame rate of the display unit.

Image generation apparatus and image generation method using frequency lower than display frame rate

A method and system of generating an image of a subject from the viewpoint of a virtual camera includes obtaining a plurality of source images of a subject and pose data for each source camera or cameras that captured the source images. Virtual camera pose data is also obtained indicating a pose of a virtual camera relative to the subject. Each source image is distorted based on a difference in pose of the corresponding source camera and the pose of the virtual camera. A weighting is determined for each distorted image based on a similarity between the pose of the corresponding source camera and the pose of the virtual camera. The distorted images are then blended together in accordance with the weightings to form an image of the subject from the viewpoint of the virtual camera.

Method and system for generating an image of a subject from a viewpoint of a virtual camera for a head-mountable display

Methods and apparatus provide for generating an image by way of acquiring information relating to at least one of a position and a rotation of a camera operated by a user. The image is generated for display on a display unit viewed by the user. A rate at which the image is generated is at a first frequency, which is lower than a second frequency corresponding to a frame rate of the display unit.

Data processing apparatus includes a camera to capture successive images according to an image capture period; an optically detectable indicator having multiple optically detectable states and configured to transition between the multiple optically detectable states according to a transition pattern having interspersed periods in the multiple optically detectable states; the transition pattern including a reference period and two or more second periods which are shorter than the image capture period and which are separated by a separation period different to the image capture period; and a processor to detect timing information from images captured by the camera of the optically detectable indicator.

Data processing

An image processing method includes detecting, for image positions in an input image, the presence of a predetermined key colour; generating an image texture, the generating step comprising: setting a texture background of the image texture to a predetermined background colour; generating transparency values at image positions of the image texture, the transparency value at an image position being dependent upon a detected property of the predetermined key colour in the input image, the transparency values; and providing a texture foreground of the image texture; and applying the image texture to the input image at image positions at which the predetermined key colour is present.

Image processing method and apparatus

Methods and apparatus for generating stereoscopic images for display with respect to a virtual display screen (VDS) include: setting a VDS position for displaying a stereoscopic image of an object in a virtual scene, the VDS having a depth profile that approximates a depth profile of the object; generating the stereoscopic image for display at the VDS within the virtual scene; and controlling a configuration of the VDS based on the detected change in the viewpoint of the stereoscopic image by the viewer.

Head-mountable display system

A video processing method includes detecting, as a reference pose, a pose of an individual at a reference time point in an input video sequence; at a second, different, time point in the input video sequence, detecting a second pose of the individual; generating from one or more source images of the individual, a transitional video sequence representing a transition of the individual from the second pose to the reference pose; and associating the transitional video sequence with the input video sequence to generate an output video sequence including at least the transitional video sequence to implement a non-linear replay branch from the second time point to the reference time point.

Video processing

A method of generating stereoscopic images for display with respect to a virtual display screen (VDS) position, includes setting a VDS position for a stereoscopic image for display such that the depth profile of one or more VDSs approximates the depth profile of one or more objects in a virtual scene to be displayed by that stereoscopic image, and generating the stereoscopic image for display having image parallax appropriate to the VDS position or positions set by the setting step.

A virtual reality apparatus includes a head mountable display (HMD) to display video content; an orientation detector to detect a current orientation of the HMD; and a display generator to generate video content for display by the HMD, the video content representing a portion of a virtual environment, the display generator selecting the portion for display by the HMD in dependence upon: a current orientation of the HMD; and a location within the virtual environment of a feature of interest.

Virtual reality

Methods and apparatus provide for receiving an image to be displayed on the head-mounted display unit generated in a first frequency by an image generation apparatus using information relating to at least one of a position and a rotation of the head of a user who wears a head-mounted display unit, where the information is acquired at a certain point of time; correcting the image using updated information relating to at least one of the position and the rotation at a different point of time, where the receiving and correcting includes receiving updated information relating to at least one of a position and a rotation in a second frequency corresponding to a frame rate of the head-mounted display unit, and performing multiple times of correction of the image in the first frequency using a plurality of the updated information relating to at least one of the position and the rotation received in the second frequency higher than the first frequency.

Image generation apparatus and image generation method

A method of generating an image of a subject in a scene includes obtaining a first image and a second image of a subject in a scene, each image corresponding to a different respective viewpoint of the subject, each image being captured by a different respective camera, where at least some of the subject is occluded in the first image and not the second image by virtue of the different viewpoints, obtaining camera pose data indicating a pose of a camera for each image, re-projecting, based on the difference in camera poses associated with each image, at least a portion of the second image to correspond to the viewpoint from which the first image was captured, and combining the re-projected portion of the second image with at least some of the first image so as to generate a composite image of the subject from the viewpoint of the first image, the re-projected portion of the second image providing image data for at least some of the occluded part or parts of the subject in the first image.

Method and system for generating an image of a subject in a scene

Methods and apparatus provide for acquiring information relating to at least one of a position and a rotation of a user's head wearing a head-mounted display; generating an image for the head-mounted display using information relating to at least one of a position and a rotation acquired at a certain point of time; and a correcting process includes receiving updated information relating to at least one of a position and a rotation at a different point of time and correcting the image generated using the updated information. The generating the image includes using the information relating to at least one of the position and the rotation in a first frequency; and the correcting includes receiving updated information relating to at least one of a position and a rotation in a second frequency corresponding to a frame rate of the head-mounted display. The correcting includes multiple times of correction of the image generated in the first frequency using a plurality of the updated information in the second frequency, which is higher than the first frequency.

Aspects of the technology encompass image encoding, which includes generating image content according to a viewpoint defined by image viewpoint data. Successive output images are generated such that each output image includes image content generated according to one or more viewpoints, and encoding metadata associated with each output image which indicates each viewpoint relating to image content contained in that output image, and which defines which portions of that output image were generated according to each of those viewpoints. An image display method for generating successive display images from successive input images is also provided, which includes re-projecting portions of each input image to form a respective display image, according to any differences between a desired display viewpoint and a particular viewpoint defined for that portion by metadata associated with a respective input image. The metadata indicating viewpoints relating to image content contained in the respective input images.

Image encoding and display

An image encoding method includes, for each of left and right images of an image stereo pair, identifying a region of interest (ROI) in an input image, generating multiple representations of that input image such that one representation is a high field of view (FOV) representation including substantially all of the input image and another representation is a low FOV representation having a lower FOV than the high FOV representation and including at least the ROI. The relationship between pixel resolution and image size for the high and low FOV representations is such that, in relation to the ROI, the low FOV representation provides more image detail than the high FOV representation. The method also includes generating one or more multiplexed images by juxtaposing portions of the left and right, high and low FOV representations so as to include all image content thereof in fewer than four multiplexed images.

A head mountable display (HMD) includes a camera operable to capture images of a peripheral and/or control device in use by a wearer of the HMD. A detector of the HMD is configured to detect occlusions in a captured image of the peripheral and/or control device. And an image renderer of the HMD is configured to render a virtual version of the peripheral and/or control device for display to the HMD wearer and to render a representation of a user's hand at a position of a detected occlusion.

Growing community of inventors

London, United Kingdom

Ian Henry Bickerstaff