Sony Interactive Entertainment Inc.

Sony Computer Entertainment Inc.

Sony Computer Entertainment America LLC

Sony Interactive Entertainment America LLC

Noam Rimon

Xiaodong Mao

Jeffrey Roger Stafford

David Perry

Rui Filipe Andrade Pereira

Xiaoyong Ye

Xiadong Mao

Eric James Larsen

Edgar Allan Tu

Frederick Umminger

Sheng Dong

Xiaodong (Crusoe) Mao

Xia Lou

Michael Reitmeyer

Jeppe Oland

A device for interfacing with an interactive program is provided, including: a housing, including, an IR projector for projecting an IR pattern into an interactive environment; a camera for capturing images of the projected IR pattern; an image processor configured to process the captured images for identification of the projected IR pattern; a transceiver configured to communicate with a computing device, the transceiver configured to transmit data generated by the image processor to the computing device, such that a position of the device in the interactive environment is determined based on the captured images of the IR pattern, the determined position of the device being utilized as input for the interactive program.

Device for interfacing with a computing program using a projected pattern

Machine learning (ML) and image recognition are used to identify an area in a presentation such as a slide presentation which is not occupied by important content. Discovery is made of background images or data that is common between multiple slides, which implies that such common content is a banner or other element that is not unique to the current slide, in turn implying that it is not critical content. In this way, images from a camera sought to be overlaid on the presentation can be automatically relocated and, if desired, resized on a slide by slide basis to areas of the non-critical content. Manual adjustment can be used for reinforcement learning of ML for future slides.

Using machine learning and image recognition for automatic relocation of camera display area and sizing of camera image

A method for sharing recorded content of an interactive application, including: recording, by a computer, a three-dimensional video clip to a storage, the three-dimensional video clip generated from a session of the interactive application; receiving, by the computer, a request to share the three-dimensional video clip over a communications network; generating, by the computer, a two-dimensional video clip by converting the three-dimensional video clip to a two-dimensional format, the two-dimensional video clip being configured for viewing through a two-dimensional display; sharing, by the computer, the two-dimensional video clip over the communications network.

Sharing recorded gameplay

A device for interfacing with an interactive program, including: a housing, including, an IR projector for projecting an IR pattern into an interactive environment; a camera for capturing images of the projected IR pattern; an image processor configured to process the captured images for identification of the projected IR pattern; a transceiver configured to communicate with a computing device that executes the interactive program, the transceiver configured to transmit data generated by the image processor to the computing device, such that a position of the device in the interactive environment is determined based on the captured images of the IR pattern, the determined position of the device being utilized to update a state of the executing interactive program at the computing device.

A system and method of using a peripheral device for interfacing with a virtual reality scene generated by a computer for presentation on a head mounted display. The peripheral device includes a haptic device capable of being placed in contact with a user and a haptic feedback controller for processing instructions for outputting a haptic signal to the haptic device. The haptic feedback controller receiving the instructions from the computer so that haptic feedback of the haptic device changes to correspond to a user's virtual interactions with a virtual object in the virtual reality scene as presented on the head mounted display.

Dynamic gloves to convey sense of touch and movement for virtual objects in HMD rendered environments

A device for enabling user control for interfacing with a video game is provided, including the following: a projector for projecting a radiation pattern; a camera for capturing images of the projected radiation pattern; an image processor configured to process the captured images for identification of the projected radiation pattern; a transceiver configured to communicate with a game console that executes the video game, the transceiver configured to transmit data generated by the image processor to the game console, such that a position of the device is determined based on the captured images of the radiation pattern, the determined position of the device being utilized to update a state of the executing video game at the game console; a harness for attaching the device to the user.

A method for sharing gameplay of a video game is provided, including: recording, by a computer, a two-dimensional (2-D) video clip to a storage, the 2-D video clip generated from gameplay of a video game; receiving, by the computer, a request to share the 2-D video clip to a social network; generating, by the computer, a three-dimensional (3-D) video clip by converting the 2-D video clip to a 3-D format, the 3-D video clip being configured for viewing through an HMD; sharing, by the computer, the 3-D video clip to the social network.

A system for interfacing with an interactive program is provided, including: a computing device for executing the interactive program, the interactive program being stored on a memory, the computing device enabling user control and input; a display device for rendering image content associated with an interactive program, the display device being configured to be attached to the user; wherein the computing device is configured to receive data from an image capture device to determine and track a position of the display device; wherein the computing device is configured to define at least two interactive zones, each interactive zone being defined by a spatial region configured to change a function of the display device when the display device is moved between the at least two interactive zones; and wherein the computing device is configured to set the function of the display device.

Systems and methods for determining functionality of a display device based on position, orientation or motion

Inertial sensors within a head mounted display are used to track movement of the head mounted display. The tracked movement of the head mounted display is correlated to an action within a virtual reality scene that is currently displayed to a user wearing the head mounted display. The action within the virtual reality scene is based on a context of the virtual reality scene that is currently displayed. The detected movement of the head mounted display can be combined with other sensor data, such as gaze detection data, to determine the action within the virtual reality scene. In this manner, movements of the user as detected using the inertial sensors within the head mounted display are used as inputs to cause actions within the current context of the virtual reality scene as displayed within the head mounted display to the user.

Methods and systems for user interaction within virtual reality scene using head mounted display

A method for presenting text information on a head-mounted display is provided, comprising: rendering a view of a virtual environment to the head-mounted display; tracking an orientation of the head-mounted display; tracking a gaze of a user of the head-mounted display; processing the gaze of the user and the orientation of the head-mounted display, to identify a gaze target in the virtual environment towards which the gaze of the user is directed; receiving text information for rendering on the head-mounted display; presenting the text information in the virtual environment in a vicinity of the gaze target.

Display of text information on a head-mounted display

A method for sharing three-dimensional gameplay of a video game, that is rendered to a head-mounted display (HMD), to a social network, is provided, including the following method operations: recording, by an HMD connected computer, a three-dimensional (3-D) video clip to a storage, the 3-D video clip generated from 3-D gameplay of a video game that is driven by interactive use of the HMD; receiving, by the HMD connected computer, a request to share the 3-D video clip to a social network; generating, by the HMD connected computer, a two-dimensional (2-D) video clip by converting the 3-D video clip to a 2-D format; sharing, by the HMD connected computer, the 2-D video clip to the social network.

Sharing three-dimensional gameplay

A method for determining the position of a controller device, comprises: receiving dimensions of the display input by a user of the computer-based system; capturing successive images of the display at the controller device; determining a position of the controller device relative to the display based on the dimensions of the display and a perspective distortion of the display in the captured successive images of the display; providing the determined position of the controller to the computer-based system to interface with the interactive program to cause an action by the interactive program.

Controller for interfacing with a computing program using position, orientation, or motion

A first application executes on one or more computing systems to generate a series of original frame pixel data sets for rendering of graphical images associated with the first application execution. A second application executes on one or more computing systems to detect one or more key graphical images present in the series of original frame pixel data sets. Upon detection of the one or more key graphical images, the second application directs replacement of a portion of pixel data in the series of original frame pixel data sets with substitute pixel data so as to generate a series of modified frame pixel data sets for encoding in lieu of the series of original frame pixel data sets.

Systems and methods for cloud processing and overlaying of content on streaming video frames of remotely processed applications

A system for interfacing with an interactive program is provided, including: a computing device for executing the interactive program; a display device for enabling user control and input to the interactive program, the display device being configured to be attached to the user; wherein the computing device is configured to receive data from an image capture device to determine and track a position of the display device; wherein the computing device is configured to define interactive zones, each interactive zone being defined by a spatial region having an associated specified function for an action of the display device when the display device is positioned within that interactive zone; and, wherein the computing device is configured to set the functionality of the action of the display device to the specified function associated with the interactive zone within which the display device is located.

A method for sharing three-dimensional gameplay of a video game to a social network is provided, the method including: receiving a request to share a recorded gameplay portion to a social network, the recorded gameplay portion defined by three-dimensional recorded video; generating a two-dimensional video clip based on the three-dimensional recorded video; sharing the two-dimensional video clip to the social network.

Growing community of inventors

Redwood City, CA, United States of America

Noam Rimon