Sony Interactive Entertainment Inc.

Jorge Arroyo Palacios

Steven Osman

Olga Rudi

Mahdi Azmandian

Jeffrey Roger Stafford

Justice Adams

Mario M Sarria, Jr

Richard Lee Marks

Todd Tokubo

Yutaka Yokokawa

Nirmal Rajkumar

Ravi Coimbatore Madhavan

Tooba Ahsen

Michael Chow

Chockalingam Ravi Sundaram

Michael Hardisty

Mark Anthony

Sandeep Bansal

Methods and system for providing streaming content of a video game at a client device includes receiving frames of streaming content from a game server. The frames represent a current game state. The frames are analyzed to generate predicted frames that are likely to occur following the current frames. The predicted frames are stored in a prediction frame buffer and used to fill any gaps in subsequent frames representing subsequent game state received from the game server.

Filling in frames using prediction for when connection drops

Techniques are described for smooth switchover of computer game control. The current states of game input is communicated to a new player assuming control. The new player is allowed time to catch up to the game. New player control is detected, and any errors are communicated to the new player. If there are differences between the old control scheme and that of the new player, they are reconciled. The outgoing player is adjusted to the transition.

Smooth switchover of computer game control

To protect a user's privacy by reducing a malicious developer's ability to eavesdrop on unwitting HMD users by converting signals from a motion sensor in the HMD to speech or speaker recognition, a microphone can record ambient sound and voice which is subtracted from the motion sensor data before the sensor data is made available to the game developer. In another technique, a band pass filter subtracts frequency in the sensor signals within the voice range. Still a third technique blends statistical noise into the motion sensor signal before passing to game developers to obfuscate the user's speech. The amount by which voice components in the motion signal are eliminated or obfuscated can be tuned by a person or app.

Tunable filtering of voice-related components from motion sensor

A method, and system for controlling a computer program are disclosed. One or more images are presented on a display device in response to execution of the computer program on a computing device. Gaze tracking information corresponding to a location of a user's gaze with respect to the one or more images presented on the display device is collected over time. The gaze tracking information over time is analyzed to detect a pattern of the user's gaze characteristic of reading and upon detecting the pattern of the user's gaze characteristics of reading, a reading control mode is activated. In the reading control mode location and duration of the user's gaze triggers commands affecting execution of the computer program.

Filtering of gaze tracking information to trigger reading control mode

Privacy of a conversation between a computer game player and an entity such as a person or virtual assistant is preserved by various techniques so that other players of the computer game who are not part of the conversation cannot apprehend the conversation. The conversation may be voice or txt but is not an online chat associated with the computer game.

SMS, phone and video call support while gaming

A method for modifying user sentiment is described. The method includes analyzing behavior of a group of players during a play of a game. The behavior of the group of players is indicative of a sentiment of the group of players during the play of the game. The method includes accessing a non-player character (NPC) during the play of the game. The NPC has a characteristic that influences a change in the sentiment of the group of players. The method includes placing the NPC into one or more scenes of the game during the play of the game for a period of time until the change in the sentiment of the group of players is determined. The change in the sentiment of the group of players is determined by analyzing of the behavior of the group of players during said play of the game.

Systems and methods for modifying user sentiment for playing a game

Methods and systems are provided for verifying an input provided at a controller including detecting a finger gesture on a surface of the controller. Responsive to detecting the finger gesture, multi-modal data is collected from a plurality of sensors and components tracking the finger gesture. The multi-modal data is used to generate an ensemble model using machine learning algorithm. The ensemble model is trained in accordance to training rules defined for different finger gestures. An output is identified from the ensemble model for the finger gesture. The output is interpreted to define an input for an interactive application selected for interaction.

Extend the game controller functionality with virtual buttons using hand tracking

A method for communicating with users in a virtual reality space includes capturing images of a user during a session of interaction in the virtual reality space, with the captured images including hand gestures and facial expressions made by the user during a communication act. The method also includes processing state data generated during the session, to identify a context associated with the virtual reality space during the communication act. The method further includes generating an audio output representing an interpretation of the hand gestures and facial expressions. The audio output includes sound modifications that are dynamically applied based on the context of virtual reality space. Still further, the method includes presenting the audio output in the virtual reality space via an avatar of the user. The face of the avatar is correspondingly modified with a video overlay associated with the sound modifications, which update during the communication act.

Animation of hand-finger communicator with real-world voice output

A method for augmenting a real-world scene viewed through augmented reality (AR) glasses includes determining that an overlay should be processed for a real-world person being viewed via the AR glasses, with the determining using artificial intelligence (AI) to identify a trigger scenario. The AI is configured to process a video stream of images captured of the real-world person using a camera of the AR glasses to identify the trigger scenario, which is associated with an intensity level exhibited by the real-world person. The method also includes identifying the overlay to replace a portion of the real-world person in the video stream of images, and generating an augmented video stream that includes the video stream composited with the overlay. The augmented video stream being presented via the AR glasses is configured to adjust the intensity level exhibited by the real-world person when viewed via the AR glasses.

Dynamic augmentation of stimuli based on profile of user

A player who is interrupted during play of a computer simulation such as a computer game and must attend to other matters (colloquially referred to herein as being 'away from keyboard' or “AFK”) is assisted in three phases. A first type of assistance is rendered when entering AFK, as in, at the start of the interruption. A second type of assistance is rendered during AFK, while the player is away. A third type of assistance is rendered when the player returns from AFK to the computer simulation.

Away from keyboard (AFK) helper for game players

To improve the fidelity of a motion sensor, voice-induced components in signals from the motion sensor as well as haptic-induced components in signals from the motion sensor are canceled prior to outputting the final motion signal to an app requiring knowledge of device motion, such as motion of a HMD for a computer game.

Fidelity of motion sensor signal by filtering voice and haptic components

Methods and systems are provided for generating augmented reality (AR) scenes where the AR scenes can be adjusted to modify at least part of an image of the physical features of a user to produce a virtual mesh of the physical features. The method includes generating an augmented reality (AR) scene for rendering on a display for a user wearing AR glasses, the AR scene includes a real-world space and virtual objects overlaid in the real-world space. The method includes analyzing a field of view into the AR scene from the AR glasses; the analyzing is configured to detect images of physical features of the user when the field of view is directed toward at least part of said physical features of the user. The method includes adjusting the AR scene, in substantial real-time, to modify at least part of the images of the physical features of the user when the physical features of the user are detected to be in the AR scene as viewed from the field of view of the AR glasses, wherein said modifying includes detecting depth data and original texture data from said physical features to produce a virtual mesh of said physical features; the virtual mesh is changed in size and shape and rendered using modified texture data that blends with said original texture data. In one embodiment, the modified physical features of the user appear to the user when viewed via the AR glasses as existing in the real-world space. In this way, when the physical features of a user are detected to be in the AR scene, the physical features are augment in the AR scene which can result in the self-perception of the user improving which in turn can provide the user with confidence to overcome challenging tasks or obstacles during the gameplay of the user.

Augmented reality artificial intelligence enhance ways user perceive themselves

Systems and methods for facilitating secret communication between players are described. One of the methods includes receiving image data of a first player to identify one or more gestures and receiving input data indicating a response of a second player to the one or more gestures. The method further includes training a model using the image data and the input data to generate an inference of communication between the first user and the second user. The method also includes generating additional inferences of communication between the first user and the second user. The method includes generating a recommendation to the second user based on the inference of communication and the additional inferences of communication.

Systems and methods for facilitating communication between players during game play

Methods and systems describe augmenting image content presented on a mirror display. The method includes detecting images of a user in an activity volume in front of the mirror display; the detecting includes identifying a body of the user and tracking movements by the user while in the activity volume. The method includes receiving a request to predict a change in physique of the body of the user, the request includes input parameters that identify a physical exercise and an amount of said physical exercise. The method includes generating an augmented image of the body of the user; the augmented image of the body has said change in physique, the change in physique of the body is assembled from one or more three-dimensional (3D) models of one or more body parts of the body that have been adjusted in shape.

Augmented image adjustment of user based on possible actions or non-actions

A system that teaches players gestures, for instance during the introduction of the game, and asks the player to invoke the gesture. Rather than asking the player to repeat over and over until the player succeeds, the game looks for commonality in the player's attempts, and after a small number of attempts, the game can learn how that player interprets the gesture given the player's own ability. The game can then adapt itself to look for that pattern to trigger the action.

Growing community of inventors

San Mateo, CA, United States of America

Jorge Arroyo Palacios