Sony Interactive Entertainment Inc.

Sony Computer Entertainment Inc.

Jeffrey Roger Stafford

Andrew Young

Steven Osman

Todd Tokubo

Glenn T Black

Chris Ho

Yunpeng Zhu

Eric James Larsen

Michael G Taylor

Anton Mikhailov

Javier Fernandez Rico

Frederick Umminger

Richard Lee Marks

Xiadong Mao

Yutaka Yokokawa

Takeo Matsukawa

Dominic Saul Mallinson

Xiaodong Mao

Jorge Arroyo Palacios

Sergey Bashkirov

Brian M Watson

Crusoe Xiadong Mao

Noam Rimon

Xiaoyong Ye

Patrick Connor

Yuichiro Nakamura

Michael Taylor

Naoki Ogishita

Mario M Sarria, Jr

Justice Adams

Koichi Obana

Javier Fernandez-Rico

Nirmal Rajkumar

Christopher Norden

Jeppe Oland

Xia Lou

Michael Chow

A sunglasses-style head worn display includes a curved waveguide with double reflective coatings/films and containing an array of light wavelength converting phosphor pinhole-size discs covered with pinhole-sized micro lenses, presenting a wide FOV virtual image for AR/VR.

Mixed reality optical system using micro phosphor and lens element array in a curved waveguide

A hand-gesture tracking engine includes a first artificial intelligence (AI) model component configured and trained to automatically identify a hand-gesture made by a source player within a video game. A hand-gesture intent engine includes a second AI model component configured and trained to automatically determine a message for communication to a target player within the video game as conveyed by the hand-gesture identified by the hand-gesture tracking engine. A hand-gesture translation engine having a third AI model component configured and trained to automatically generate a communication to the target player that conveys the message as determined by the hand-gesture intent engine. The third AI model component is configured and trained to automatically control sending of the communication to the target player.

Systems and methods for artificial intelligence (AI)-assisted temporal context processing in support of hand-gesture communication in video gaming

Methods and systems for determining a type of material of an object in a real-world environment are described. One of the methods includes receiving a plurality of sets of audio data based on sounds received from a plurality of objects within a plurality of environments. The method further includes receiving a plurality of sets of input data regarding a plurality of types of materials of the plurality of objects, training an artificial intelligence (AI) model based on the plurality of sets of audio data and the plurality of sets of input data, and applying the AI model to a set of audio data captured from the real-world environment to determine the type of material of the object within the real-world environment to enhance the realism of augmented reality (AR) or virtual reality (VR) video games and applications.

Systems and methods for determining a type of material of an object in a real-world environment

A method for assisting game play. The method includes monitoring game play of the user playing a gaming application, wherein the user has a defined task to accomplish, wherein the task is associated with a task type. The method includes determining a task type proficiency rule for the task type based on results of a plurality of players taking on a plurality of tasks having the task type. The method includes determining a player proficiency score for accomplishing the task based on the task type proficiency rule. The method includes determining a user predictive rate of success in accomplishing the task based on the player proficiency score, the task type proficiency rule, and the task. The method includes determining a recommendation for the user based on the user predictive rate of success.

Automated artificial intelligence (AI) personal assistant

Systems and methods for determining movement of a hand-held controller (HHC) with respect to a head-mounted display (HMD) are described. A camera is located at a lower region of the HMD for capturing images of the HHC. The images of the HHC are used to determine a position and an orientation of the HHC with reference to the HMD. Also, the camera or another camera can be used to determine a position and orientation of the HMD with respect to a real-world environment. The position and orientation of the HMD and the position and orientation of the HHC are used to change to state of an application executed on a game cloud.

Systems and methods for determining movement of a controller with respect to an HMD

Gaze tracking data representing a user's gaze is analyzed to determine one or more regions of interest. One or more gaze tracking parameters are determined from the gaze tracking data. Adjusted foveation data is determined representing an adjusted size and/or shape of one or more regions of interest in one or more images to be subsequently presented to the user based on the one or more gaze tracking parameters. The compression of the one or more transmitted images is adjusted so that fewer bits are needed to transmit data for portions of an image outside the one or more regions of interest than for portions of the image within the one or more regions of interest. Adjusting compression includes eliminating the region(s) of interest from images that are presented to the user during the saccade or blink.

Use of eye tracking to adjust region-of-interest (ROI) for compressing images for transmission

A method for gaming. The method includes assigning a default game play profile to a user, wherein the default game play profile includes a default game play style that simulates human game play, and wherein the default game play profile is configured to control game play for the user based on the default game play style. The method includes monitoring a plurality of game plays of the user playing a plurality of gaming applications. The method includes generating a user game play profile of the user by adjusting the default game play style based on the plurality of game plays, wherein the user game play profile incudes a user game play style customized to the user. The method includes controlling an instance of a first gaming application based on the user game play style of the user game play profile.

Generation of an artificial intelligence (AI) model for automatic control of characters during game plays of gaming applications

By using the user head and/or eye tracking capabilities of an extended reality (XR) system, a region of gaze (ROG) of the user towards the virtual screen can be determined and signaled to non-XR games and applications running on the host system connected to the XR system. The existing non-XR games/applications can be patched, upgraded, or remastered to support the application of the ROG and ROG based enhancements of the visuals and user interfaces, using the provided by the system software on the host system that is rendering the game for the XR system.

Foveated enhancement of non-XR games within a HMD system

Systems and methods for facilitating secret communication between players are described. One of the methods includes receiving image data of a first player to identify one or more gestures and receiving input data indicating a response of a second player to the one or more gestures. The method further includes training a model using the image data and the input data to generate an inference of communication between the first user and the second user. The method also includes generating additional inferences of communication between the first user and the second user. The method includes generating a recommendation to the second user based on the inference of communication and the additional inferences of communication.

Systems and methods for facilitating communication between players during game play

Alpha values from an image frame buffer are used to control one or more of a transparent display and an occlusion screen adjacent to the transparent display to render images with coloring perceptible to a person viewing the transparent display.

Alpha value use to render images on transparent display with colors perceptible to a person

AR glasses are provided with left and right temples extending well behind the ears of the wearer to establish a center of gravity of the glasses that is approximately vertically above the front portion of the ears of the wearer. Rear segments the temples may be detachably engaged with the temples and may hold one or more electrical components.

Balanced augmented reality glasses with augmentation attachments

Methods and systems for filtering content include identifying content for presenting in a rendering space defined by one or more mechanisms of a head mounted display (HMD), wherein the content is interactive, streaming content. A rating score for the content is identified. The content is dynamically adjusted to selectively filter a portion of the content based on a user profile of the user, rating score of the content, and ongoing interactions detected at the content. The adjusted content is formatted for rendering on the HMD. The formatted content is automatically transmitted to the HMD for rendering, in response to the request.

Control of personal space content presented via head mounted display

Methods and systems are provided for generating an outcome of a previous gameplay. The method includes providing access of a recording of the previous gameplay, the recording is one that was shared by a player for the previous gameplay. The method includes providing an interface that identifies one or more what-if scenarios for the previous gameplay. The one or more what-if scenarios being selected for inclusion into the interface is based on processing the recording of the previous gameplay through a machine learning model to identify what-if scenarios for the user, and options for what-if scenarios are provided to the interface. The method includes enabling the selection of at least one of the what-if scenarios from the options. The selection causes an adjustment to metadata associated with the recording of the previous gameplay. The method includes replaying the recording that was selected using the adjusted metadata to generate the alternative outcome of the previous gameplay.

Interactive what-if game replay methods and systems

Gaze tracking data representing a user's gaze is analyzed to determine one or more regions of interest. One or more gaze tracking error parameters are determined from the gaze tracking data including a rate of rotation of a user's eye with respect to one or more axes. An error in a fixation of the user's eye on a particular region of interest is determined. Adjusted foveation data is determined representing an adjusted size and/or shape of one or more regions of interest in one or more images to be subsequently presented to the user based on the one or more gaze tracking parameters. The compression of the one or more transmitted images is adjusted so that fewer bits are needed to transmit data for portions of an image outside the one or more regions of interest than for portions of the image within the one or more regions of interest.

A method to identify positions of fingers of a hand is described. The method includes capturing images of a first hand using a plurality of cameras that are part of a wearable device. The wearable device is attached to a wrist of a second hand and the plurality of cameras of the wearable device is disposed around the wearable device. The method includes repeating capturing of additional images of the first hand, the images and the additional images captured to produce a stream of captured image data during a session of presenting the virtual environment in a head mounted display (HMD). The method includes sending the stream of captured image data to a computing device that is interfaced with the HMD. The computing device is configured to process the captured image data to identify changes in positions of the fingers of the first hand.

Growing community of inventors

Redwood City, CA, United States of America

Jeffrey Roger Stafford