Sony Interactive Entertainment Inc.

Sony Interactive Entertainment Europe Limited

Nigel John Williams

Fabio Cappello

Andrew William Walker

Rosario Leonardi

Rajeev Gupta

Maurizio Cerrato

Mark Jacobus Breugelmans

Gregory James Bedwell

Oliver George Hume

Patrick John Connor

Matthew William Sanders

Timothy Edward Bradley

Richard James Forster

Mandana Jenabzadeh

Jessica Rose Grant

Robert Christopher Starkey

Jake Leigh Clarke

Methods and/or apparatus provide for obtaining intrinsics and extrinsics of a plurality of cameras arranged to capture a scene and based thereon, generate a voxel representation of the scene; obtaining extrinsics and intrinsics of a virtual camera; mapping the extrinsics and intrinsics of the virtual camera to at least one voxel surface defined in a data structure and to identify at least one camera pair suitable for obtaining depth data for that voxel surface based on the corresponding camera pair identifier; and receiving an input from the mapping unit, and in response obtaining images captured by the cameras in the at least one camera pair identified as being suitable for reconstructing the scene from the viewpoint of the virtual camera, and to generate an image of the scene from the viewpoint of the virtual camera based on the obtained images.

Free-viewpoint method and system

A system for improving a discovery process for a virtual environment, the system comprising an environment discovery unit operable to perform a discovery process comprising navigation of the virtual environment and identification of one or more aspects of the virtual environment, a performance analysis unit operable to evaluate the effectiveness of the discovery process, and a discovery update unit operable to modify future operation of the environment discovery unit in dependence upon the evaluation.

System, method and storage medium for identification of one or more aspects of a virtual environment

A method of generating a three-dimensional (3D) reconstruction of a human includes obtaining at least one colour image and corresponding depth image of a scene comprising a human subject that is at least partially occluded by one or more items. The regions in the at least one colour image corresponding to a human subject are identified. A point cloud of the scene is generated based on the at least one colour image and corresponding depth image. One or more parameters of a parametric model are then adjusted based on the regions of the point cloud identified as corresponding to the human subject. The adjusted parametric model is then used to generate a 3D reconstruction of the human subject; where the parametric model provides an estimate for points in the point cloud corresponding to the occluded parts of the human subject. A corresponding system is also provided.

Method and system for generating a 3D reconstruction of a human

An encoding apparatus is provided. The apparatus comprises an input unit operable to obtain a plurality of training images, said training images being for use in training a machine learning model. The apparatus also comprises a label unit operable to obtain a class label associated with the training images; and a key unit operable to obtain a secret key for use in encoding the training images. The apparatus further comprises an image noise generator operable to generate, based on the obtained secret key, noise for introducing into the training images. The image noise generator is configured to generate noise that correlates with the class label associated with the training images such that a machine learning model subsequently trained with the modified training images learns to associate the introduced noise with the class label for those images. A corresponding decoding apparatus is also provided.

Encoding and decoding apparatus

A system for adjusting the pose of a camera relative to a subject in a scene is provided. The system comprises a camera operable to capture images of a scene; an identification unit configured to identify an object of interest in images of the scene; a pose processor configured to obtain a pose of the object of interest in the scene relative to the camera; a scene analyser operable to determine, based on at least one of the obtained pose of the object of interest and images captured by the camera, a scene quality associated with images captured by the camera. A controller is configured to cause the pose of the camera to be adjusted based on a determination that the scene quality of an image captured at a current pose is less than a threshold value. A corresponding device is also provided.

System and camera device for capturing images

An apparatus includes a video display to display video images to a user; a gaze detector configured to detect a gaze direction for one or both eyes of the user while the user views the display; a head tracker configured to detect a head orientation of the user; an image processor configured to generate the video images for display by the video display; the image processor being responsive to one or more control functions dependent upon the gaze direction detected by the gaze detector; and a controller configured to detect a predetermined condition and, in response to detection of the predetermined condition, to control the image processor to be responsive to one or more control functions dependent upon the head orientation detected by the head tracker in place of the one or more control functions dependent upon the gaze direction detected by the gaze detector.

Video processing

A method of generating a 3D reconstruction of a scene, the scene comprising a plurality of cameras positioned around the scene, comprises: obtaining the extrinsics and intrinsics of a virtual camera within a scene; accessing a data structure so as to determine a camera pair that is to be used in reconstructing the scene from the viewpoint of the virtual camera; wherein the data structure defines a voxel representation of the scene, the voxel representation comprising a plurality of voxels, at least some of the voxel surfaces being associated with respective camera pair identifiers; wherein each camera pair identifier associated with a respective voxel surface corresponds to a camera pair that has been identified as being suitable for obtaining depth data for the part of the scene within that voxel and for which the averaged pose of the camera pair is oriented towards the voxel surface; identifying, based on the obtained extrinsics and intrinsics of the virtual camera, at least one voxel that is within the field of view of the virtual camera and a corresponding voxel surface that is oriented towards the virtual camera; identifying, based on the accessed data structure, at least one camera pair that is suitable for reconstructing the scene from the viewpoint of the virtual camera, and generating a reconstruction of the scene from the viewpoint of the virtual camera based on the images captured by the cameras in the identified at least one camera pair.

Apparatus comprises an image processor configured to provide output video images to a head mountable display, HMD, having one or more display elements to display video images to a wearer of the HMD, for display by the one or more display elements in response to input video images, in which each output video image corresponds to a respective input video image; in which: the image processor comprises a detector configured to detect whether an input image brightness at image locations in the input video images exceeds a threshold image brightness; and the image processor is configured to vary a relationship between the display properties at a given image location in a given output video image and the input image display properties at the given image location in the corresponding input video image in response to a detection that image brightness at the given image location in one or more input video images preceding that corresponding input video image exceeded the threshold image brightness.

A system for modifying data representing a virtual environment includes: an environment navigation unit operable to control navigation within the virtual environment to generate one or more viewpoints within the virtual environment, an environment identification unit operable to identify one or more aspects of the geometry of the virtual environment visible in the one or more viewpoints, a geometry evaluation unit operable to evaluate the visibility of one or more aspects of the geometry based upon the identification for each of one or more viewpoints, and a data modification unit operable to modify one or more elements of data representing the virtual environment.

System and method for modifying content of a virtual environment

A system for controlling interactions between a plurality of real and virtual robots, includes one or more real robots present in the real environment, one or more virtual robots present in a virtual environment corresponding to the real environment, and a processing device operable to control interactions between one or more of the real robots and one or more of the virtual robots, where the interactions between the real and virtual robots are dependent upon at least the positions of the one or more real robots in the real environment and the positions of the one or more virtual robots in the virtual environment.

Apparatus control systems and method

A digital model repair method includes: providing a point cloud digital model of a target object as input to a generative network of a trained generative adversarial network 'GAN', the input point cloud comprising a plurality of points erroneously perturbed by one or more causes, and generating, by the generative network of the GAN, an output point cloud in which the erroneous perturbation of some or all of the plurality of points has been reduced; where the generative network of the GAN was trained using input point clouds comprising a plurality of points erroneously perturbed by said one or more causes, and a discriminator of the GAN was trained to distinguish point clouds comprising a plurality of points erroneously perturbed by said one or more causes and point clouds substantially without such perturbations.

Digital model repair system and method

A system for characterising surfaces in a real-world scene, the system comprising an object identification unit operable to identify one or more objects within one or more captured images of the real-world scene, a characteristic identification unit operable to identify one or more characteristics of one or more surfaces of the identified objects, and an information generation unit operable to generate information linking an object and one or more surface characteristics associated with that object.

Surface characterisation apparatus and system

Methods and apparatus provide for generating content for display including one or more objects to be rendered, including: defining a plurality of virtual elements; setting one or more parameters associated with each of the plurality of virtual elements; and generating content for display using the plurality of virtual elements and corresponding parameter values, where setting the one or more parameters includes setting one or more parameters in accordance with the output of a machine learning algorithm adapted to determine appropriate parameters for the plurality of virtual elements so as to simulate an object to be rendered.

Content generation system and method

A foveated rendering system for modifying content to be displayed includes a user profile obtaining unit operable to obtain a user profile comprising information about head motion and eye motion for a user, and a foveated rendering unit operable to apply a foveated rendering process to the content to be displayed in dependence upon obtained user profile information.

Growing community of inventors

London, United Kingdom

Nigel John Williams