Sony Interactive Entertainment Inc.

Sony Interactive Entertainment Europe Limited

Andrew James Bigos

Gilles Christian Rainer

Fabio Cappello

Norihiro Nagai

Timothy Edward Bradley

Sahin Serdar Kocdemir

Marina Villanueva Barreiro

Jason Gordon Doig

Masakazu Suzuoki

Masakazu Hayashi

Matthew William Sanders

Richard James Forster

Robert Keith John Withey

Neil Jonathan Brown

Cristian Craciun

Joseph Charles Boulter

Jean-Paul Roberts

Neil Brown

A method of generating an image of a scene from a target viewpoint is provided. The method comprises capturing a plurality of multi-plane images of a virtual scene, and determining for each multi-plane image, whether a frontmost clipping plane pair defined for that multi-plane image intersects a virtual object in the virtual scene. Responsive to a positive determination for a respective clipping plane pair, the corresponding image data in the corresponding image plane is identified and assigned an identifier, thus generating a modified version of the corresponding multi-plane image. Camera information is obtained for the one or more virtual cameras that captured the multi-plane images and the target virtual camera. An output image is then generated by combining at least one of the modified multi-plane images with another of the multi-plane images in accordance with the obtained camera information. A corresponding system is also provided.

Method and system for generating a target image from plural multi-plane images

An image rendering method for a virtual scene includes, for a plurality of IDs, generating a respective mask identifying elements of the scene that are associated with a respective ID; for the resulting plurality of masks, dividing a respective mask into a plurality of tiles; and discarding tiles that do not identify any image elements; for the resulting plurality of remaining tiles, selecting a respective trained machine learning model from among a plurality of machine learning models, the respective machine learning model having been trained to generate data contributing to a render of at least a part of an image, based upon elements of the scene associated with the same respective ID as the elements identified in the mask from which the respective tile was divided; and using the respective trained machine learning model to generate data contributing to a render of at least a part of the image based upon input data at least for the identified elements in the respective tile.

Image rendering method and apparatus

A method of generating a training set for a neural precomputed light model includes: generating a plurality of candidate viewpoints of a scene, culling candidate viewpoints according to a probability that depends upon a response of the surface of the scene to light at a surface position in the scene corresponding to the viewpoint, and generating training images at the remaining viewpoints.

An image rendering method for rendering a pixel at a viewpoint including: for a first element of a virtual scene, having a predetermined surface at a position within that scene, providing the position and a direction based on the viewpoint to a machine learning system previously trained to predict a factor that, when combined with a distribution function that characterises an interaction of light with the predetermined surface, generates a pixel value corresponding to the first element of the virtual scene as illuminated at the position, combining the predicted factor from the machine learning system with the distribution function to generate the pixel value corresponding to the illuminated first element of the virtual scene at the position, and incorporating the pixel value into a rendered image for display.

An image rendering method for rendering a pixel at a viewpoint includes, for a first element of a virtual scene, having a predetermined surface at a position within that scene; providing the position and a direction based on the viewpoint to a machine learning system previously trained to predict a factor that, when combined with a distribution function that characterises an interaction of light with the predetermined surface, generates a pixel value corresponding to the first element of the virtual scene as illuminated at the position, combining the predicted factor from the machine learning system with the distribution function to generate the pixel value corresponding to the illuminated first element of the virtual scene at the position, and incorporating the pixel value into a rendered image for display, where the machine learning system was previously trained with a training set based on images comprising multiple lighting conditions.

An image rendering method for rendering a pixel at a viewpoint includes: for a first element of a virtual scene, having a predetermined surface at a position within that scene, evaluating whether to render a pixel corresponding to the first element using a machine learning system having been trained to output a value representative of the lighting of the predetermined surface at the position, or using an alternative rendering approach, and rendering the pixel according to which of the machine learning system and the alternative rendering approach are chosen.

An image rendering method for rendering a pixel at a viewpoint includes: for a first element of a virtual scene, having a predetermined surface at a position within that scene, evaluating whether to render a pixel corresponding to the first element using at least a first machine learning system having been trained to generate an illuminance output representative of the lighting of the predetermined surface at the position, or using an alternative rendering approach, and rendering the pixel according to which of the at least first machine learning system and the alternative rendering approach are chosen in the evaluating step; where the evaluating step comprises obtaining a confidence value from the at least first machine learning system indicative of the accuracy of the illuminance output, the machine learning system having been trained to generate the confidence value in conjunction with the illuminance output, and the rendering step comprises using the alternative rendering approach if the confidence value does not satisfy a predetermined threshold value.

An image rendering method for rendering a pixel of a virtual scene at a viewpoint includes: downloading a machine learning system corresponding to a current or anticipated state an application determining the virtual scene to be rendered from among a plurality of machine learning systems corresponding to a plurality of states of the application; providing a position and a direction based on the viewpoint to the machine learning system previously trained to predict a factor; combining the predicted factor from the machine learning system with a distribution function that characterises an interaction of light with a predetermined surface to generate the pixel value corresponding to an illuminated first element of the virtual scene at the position; and incorporating the pixel value into a rendered image for display.

A method of enhancing a 3D printed model includes generating successive visualisations of a virtual environment comprising a target object, receiving a user input indicating selection of a visualisation of the target object at a particular moment in time, generating visualisation data to enable subsequent visualisation of at least the target object as at the particular moment in time, causing the visualisation data to be stored at a unique location;

System and method of enhancing a 3D printed model

A method of 3D print modelling includes: obtaining a target virtual object for 3D printing, evaluating the target virtual object for 3D printing with respect to one or more printability criteria to detect whether the 3D printed model would have one or more predetermined undesirable physical characteristics, and if so, generating 3D printable model data defining a transparent matrix to be printed surrounding the 3D printed model of the target virtual object.

System and method of 3D print modelling

An apparatus includes: an object data storage section that stores polygon identification data for polygons of an object to be displayed; a reference image data storage section that stores data of reference images each representing an image when a space including the object to be displayed is viewed from one of a plurality of prescribed reference viewing points, and further stores polygon identification data corresponding to each reference image; a viewing point information acquisition section that acquires information relating to a viewing point; a projection section that represents on a plane of a display image the position and shape of an image of the object when the space is viewed from the viewing point; a pixel value determination section that determines the values of pixels constituting the image of the object in the display image, using the values of the pixels representing the same image in one or more of the plurality of reference images; and an output section that outputs the data of the display image; wherein for a subject pixel, the pixel value determination section is arranged to determine the position on a reference image corresponding to the position of the subject pixel on the object, obtain the polygon identification corresponding to the determined position on the reference image, compare the obtained polygon identification with the polygon identification of the polygon corresponding to the position of the subject pixel on the object; and select the reference image if the compared polygon identifications match.

Determining pixel values using reference images

A method of 3D print modelling includes: generating a point cloud representation of a target virtual object for 3D printing, generating a voxel representation of the point cloud by adding a respective first voxel when a respective first point from the point cloud representation is located in the notional volume occupied by that first voxel, assigning a first colour associated with the first point to the faces of the first voxel, and if a second point from the point cloud representation is located within the volume of the first voxel, assigning a second colour associated with the second point to a respective face of the new voxel corresponding to the normal of the second point; and thickening the voxel representation of the point cloud by generating a duplicate voxel adjacent to the first voxel along a first axis passing through two opposing faces of the first voxel respectively having the first and second colours.

System and method of 3D print modeling utilizing a point cloud representation and generating a voxel representation of the point cloud

A method of generating a model for 3D printing, includes selecting a target object within a virtual environment; sampling the target object to form a point cloud, the point cloud comprising points corresponding to an outer surface of the target object and also one or more internal features of the target object; rendering the point cloud from a plurality of viewpoints using voxels in place of the points in the point cloud; detecting which voxels and hence which points of the cloud were rendered over the plurality of renders; forming a surface-only point cloud comprising those points of the cloud that were rendered; and generating a model for 3D printing based on the surface-only point cloud.

A flood-fill method for parallel implementation includes the steps of receiving an image including first elements having a flood fill source colour to be replaced with a flood fill target colour, and second elements not having the flood fill source colour; dividing the image according to a hierarchical space-partitioning scheme to form cells at a plurality of levels in the hierarchy; detecting the occupancy of each cell by second elements; for any cell having no occupancy by second elements, set a depth elevation value for child cells of that cell to be one greater than the value for that cell, indicating that the flood fill shall navigate levels towards a root cell by the depth elevation amount of a cell whilst only occupying nodes without any second elements, thereby reaching the largest parent cell which can be flood-filled; and filling the flood fill source colour of cells having no occupancy by second elements with the flood fill target colour, thereby modifying the received image.

Parallel method of flood filling, and apparatus

An image generating device includes a rendering section that renders test images each representing an image when a space including an object to be displayed is viewed from one of a plurality of candidate reference viewing points; a candidate reference viewing point evaluation section that evaluates an importance value for the candidate reference viewing points as a function of at least their comparative coverage of points in the space; and an update section that changes the position of one or more candidate reference viewing points that have a low importance value, obtains a re-evaluation from the candidate reference viewing point evaluation section, and does not revert the position of a candidate reference viewing point if its comparative coverage of points in the space has increased.

Growing community of inventors

Stains Upon Thames, United Kingdom

Andrew James Bigos