Sony Interactive Entertainment Inc.

Michael Taylor

Jeffrey Roger Stafford

Celeste Bean

Glenn T Black

Ruxin Chen

Sudha Krishnamurthy

Sergey Bashkirov

Erik Beran

Javier Fernandez Rico

Masanori Omote

Jaekwon Yoo

Yurika Kondo

Mihee Kang

Javier Fernandez-Rico

Arran Green

Yasunori Matsui

Methods and systems are provided for providing real world assistance by a robot utility and interface device (RUID) are provided. A method provides for identifying a position of a user in a physical environment and a surface within the physical environment for projecting an interactive interface. The method also provides for moving to a location within the physical environment based on the position of the user and the surface for projecting the interactive interface. Moreover, the method provides for capturing a plurality of images of the interactive interface while the interactive interface is being interacted with by the use and for determining a selection of an input option made by the user.

Robot utility and interface device

Alerts can be generated to inform a prime user to unblock another user, whose audio and/or video the prime user has blocked, owing to an emerging context that may be of interest to the prime user. The prime user can be informed of such a context and provided the ability to selectively view the blocked user's comments so the full context can be understood. This aims to help protect the prime user who may have been the subject of harassment from the blocked user originally causing the blocked user to be muted or otherwise blocked in the first place.

Context sensitive alerts involving muted users

A machine learning-based model is configured to make inferences about computer game actions to execute based on dynamic, varying player gestures and to translate those game actions into input sequence macros. In some instances, the button sequence mapping for the macros can even dynamically change based on game state so that different macros for the same computer game action might be inferred by the model depending on game state.

Button sequence mapping based on game state

A computer simulation object such as a chair is described by voice or photo input to render a 2D image. Machine learning may be used to convert voice input to the 2D image. The 2D image is converted to a 3D asset and the 3D asset or portions thereof are used in the computer simulation, such as a computer game, as the object such as a chair.

Voice driven modification of sub-parts of assets in computer simulations

A method including accessing an avatar including a plurality of physiological characteristics, wherein the avatar includes a plurality of motion profiles. The method including receiving editing of a motion profile of the avatar via a user interface. The method including modifying each of the plurality of motion profiles to be consistent with the motion profile that edited. The method including automatically generating a prompt to apply the motion profile that is edited to the avatar, wherein the prompt is provided as input for an image generation artificial intelligence system configured to implement latent diffusion. The method including generating a sequence of video frames using the image generation artificial intelligence system showing the avatar in motion based on the motion profile that is edited. The method including presenting the sequence of video frames in the user interface.

Avatar generation and augmentation with auto-adjusted physics for avatar motion

A method for building an artificial intelligence (AI) model. The method includes accessing data related to monitored behavior of a user. The data is classified, wherein the classes include an objective data class identifying data relevant to a group of users including the user, and a subjective data class identifying data that is specific to the user. Objective data is accessed and relates to monitored behavior of a plurality of users including the user. The method includes providing as a first set of inputs into a deep learning engine performing AI the objective data and the subjective data of the user, and a plurality of objective data of the plurality of users. The method includes determining a plurality of learned patterns predicting user behavior when responding to the first set of inputs. The method includes building a local AI model of the user including the plurality of learned patterns.

Private artificial intelligence (AI) model of a user for use by an autonomous personal companion

A 3D neural radiance field (NeRF) is used to generate an impulse response (IR) characterization, which can then be input to a convolutional reverberation engine to create an audio experience that reflects the in-game world on a 2.0 stereo speaker system. The NeRF recreates a background geometry and the impulse response of a virtual 3D space generated using NeRF is input to the convolutional reverberation engine to transform game sounds/music to appear as though they are occurring inside the 3D space of the game. The same may be done for the player's real-world space in which the virtual IR and real IR are blended together in real-time, and the real-life player tracked as he moves around the room to create audio as it would sound were the player moving through the virtual space while adjusting for the acoustics of the real space.

Transforming computer game audio using impulse response of a virtual 3D space generated by NeRF input to a convolutional reverberation engine

Machine learning is used to transform screen renders from the viewpoint of the player's character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints. To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered/sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.

Use of machine learning to transform screen renders from the player viewpoint

A method generating an image according to a style is described. The method includes receiving a textual description describing a first image. The method further includes applying an artificial intelligence (AI) model to determine the style of a second image based on the textual description, the first image, a plurality of descriptions, and a plurality of images to generate a suggestion. The style provides a context to the second image. The style is lacking in the first image. The method includes generating the second image with the suggestion according to the style and providing the second image to a client device for display.

Systems and methods for generating images to achieve a style

A method, system and computer program product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and comparing the integrated output value to a backlash threshold. A subsequent output value is generated using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. A position of the one or more movable joints is simulated based on an integral of the subsequent output value; and the Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints.

Domain adaptation for simulated motor backlash

A character in a game world of a computer simulation is identified as moving toward a sky box in the simulation. The computer simulation does not permit simulation characters to enter the sky box. Techniques are described for remastering the skybox based on various parameters.

Growing community of inventors

San Mateo, CA, United States of America

Michael Taylor