Sony Interactive Entertainment Inc.

Xiaoyong Ye

Yuichiro Nakamura

Jeffrey Roger Stafford

Warren M Benedetto

Noam Rimon

Todd Tokubo

Eric James Larsen

Mahdi Azmandian

Dennis Dale Castleman

Yutaka Yokokawa

Frederick Umminger

Xiaodong (Crusoe) Mao

Xia Lou

A tracking system uses one or more light sources to direct light toward one or more of a user's eyes. A dynamic vision sensor (DVS) operably coupled to a processor is configured to view the user's eyes. The DVS has an array of light-sensitive elements in a known configuration. The DVS outputs signals corresponding to events at corresponding light-sensitive elements in the array in response to changes in light from the light sources reflecting from a portion of the user's eyes. The output signals include information corresponding to locations of the corresponding light-sensitive elements in the array. The processor determines an association between each event and a corresponding particular light source and fits an orientation of the user's eyes using the determined association, the relative location of the light sources with respect to the user's eyes and the locations of the corresponding light-sensitive elements in the array.

Dynamic vision sensor based eye and/or facial tracking

A position and orientation of a controller are determined from a known configuration of two or more light sources with respect to each other and with respect to a controller body and from output signals from a dynamic vision sensor (DVS) generated in response to changes in light output from the light sources. The output signals indicate times events at corresponding light-sensitive elements in an array in the DVS and array locations of the light-sensitive elements. A position and orientation of one or more objects are determined from signals generated by two or more light-sensitive elements resulting from other light reaching the two or more light-sensitive elements.

Deployment of dynamic vision sensor hybrid element in method for tracking a controller and simultaneous body tracking, slam or safety shutter

A tracking system uses a dynamic vision sensor (DVS) operably coupled to the processor. The DVS has an array of light-sensitive elements in a known configuration. The DVS outputs signals corresponding to two or more events at two or more corresponding light-sensitive elements in the array in response to changes in light output from two or more light sources mounted in a known configuration with respect to each other and with respect to a controller. The output signals indicate times of the events and locations of the corresponding light-sensitive elements. The processor determines an association between each event and two or more corresponding light sources and fits a position and orientation of the controller using the determined association, the known configuration of the light sources and the locations of the corresponding light-sensitive elements in the array.

Asynchronous dynamic vision sensor LED AI tracking system and method

Methods and systems are provided for generating recommendations related to a game. One example method is for generating recommendations for a game executed by a cloud gaming service. The method includes receiving, by a server, sensor data captured during gameplay of the game by a plurality of user, and each of the plurality of sensor data includes intensity information related to reactions made by respective users. The method includes processing, by the server, features from the sensor data and the interactive data from the game when the users played the game. The features are classified and used to build an engagement model that identifies relationships between specific ones of the plurality of sensor data and the interactive data. The method includes processing, by the server, sensor data captured during a current gameplay by a user using the engagement model. The processing is configured to generate a recommendation to the user regarding an action to take to progress in the game during said current gameplay.

Human performance capturing for artificial intelligence recommendations

A DVS has an array of light-sensitive elements in a known configuration and outputs signals corresponding to events at corresponding light-sensitive elements in the array in response to changes in light output from two or more light sources in a known configuration with respect to each other and with respect to a controller body. The signals indicate times of the events and array locations of the corresponding light-sensitive elements. Filters selectively transmit light from the light sources to light-sensitive elements in the array and selectively block other light from reaching those elements and vice versa. A processor determines a position and orientation of the controller from times of the events, array locations of corresponding light-sensitive elements, and the known light source configuration and determines a position and orientation of one or more objects from signals generated by two or more light-sensitive elements resulting from other light.

Hybrid pixel dynamic vision sensor tracking using IR and ambient light (or depth sensor)

A tracking system includes a processor, a controller, two or more light sources and a dynamic vision sensor (DVS). The light sources are of known configuration with respect to each other the controller and turn on and off in a predetermined sequence. The DVS includes an array of light-sensitive elements of known configuration. The DVS outputs signals corresponding to events at corresponding light-sensitive elements in the array in response to changes in light from the light sources. The signals indicate times of the events and locations of the corresponding light-sensitive elements. The processor determines an association between each event and one or more of the light sources and, from that association, determines an occlusion of one or more of the light sources. The processor estimates a location of an object using the determined occlusion, the known light source configuration, and the locations of the corresponding light-sensitive elements in the array.

Dynamic vision sensor tracking based on light source occlusion

To track certain difficult facial features during speech such as the corners of the mouth and the teeth, a camera sensor system generates RGB/IR images and the system also uses light intensity change signals from an event driven sensor (EDS), as well as voice analysis. In this way, the camera sensor system enables improved performance tracking (equivalent to using very high-speed camera) at lower bandwidth and power consumption.

Dynamic vision sensor for visual audio processing

Detection of whether a video is a fake video derived from an original video and altered is undertaken using a block chain that either forbids adding to the block chain copies of original videos that have been altered or indicating in the block chain that an altered video has been altered. Image fingerprinting techniques are described for determining whether video sought to be added to block chain has been altered.

Fake video detection using block chain

Methods and systems for enhancing viewing experience of a spectator includes receiving a request for viewing game play of a video game executing at a game cloud server. Responsive to the request, the spectator is initially provided with one of a player camera view or an active spectator camera view generated for the game play. The system dynamically switches between the player camera view and the active spectator camera view based on context of action occurring in the player camera view and the active spectator camera view. The dynamic switching correlates with time of action occurring in the game scene of the video game.

Camera view selection processor for passive spectator viewing

A method includes using an electromagnetic (EM) tracking system to track a tangible object, detecting a presence of interference with a magnetic field generated by the EM tracking system, and compensating for the interference. A system includes an EM tracking transmitter, an EM tracking receiver, and a processor based apparatus in communication with the EM tracking transmitter and the EM tracking receiver. The processor based apparatus is configured to execute steps including using the EM tracking transmitter and the EM tracking receiver to implement an EM tracking system. A storage medium storing one or more computer programs is also provided.

Detecting and compensating for magnetic interference in electromagnetic (EM) positional tracking

To facilitate control of an AR HMD, a camera unit in a camera sensor system generates RGB/IR images and the system also extrapolates images for times in the future based on light intensity change signals from an event detection sensor (EDS) for HMD pose tracking, hand tracking, and eye tracking. The times in the future are provided by an HMD application which defines the future times, and the RGB/IR images and extrapolated images are sent back to the application. In this way, the camera sensor system enables improved performance tracking (equivalent to using very high-speed camera) at lower bandwidth and power consumption.

Combined light intensity based CMOS and event detection sensor for high speed predictive tracking and latency compensation in virtual and augmented reality HMD systems

Detection of whether a video is a fake video derived from an original video and altered is undertaken using both image analysis and frequency domain analysis of one or more frames of the video. The analysis may be implemented using neural networks.

Growing community of inventors

San Mateo, CA, United States of America

Xiaoyong Ye