Sony Interactive Entertainment Inc.

Sony Computer Entertainment Inc.

Intel Corporation

Sony Computer Entertainment America LLC

Xiaodong Mao

Noam Rimon

Gary M Zalewski

Jeffrey Roger Stafford

Richard Lee Marks

Steven Osman

Ruxin Chen

Gustavo Hernandez-Abrego

Rishi Deshpande

Baining Guo

Glenn T Black

Anton Mikhailov

Xavier Menendez-Pidal

Care Michaud-Wideman

Dominic Saul Mallinson

Todd Tokubo

Eric James Larsen

Brian M Watson

Methods, systems, and computer programs are presented for managing the sensory stimulus generated by a head mounted display (HMD). One method includes identifying a game state of a game being executed for a current session by a computer for display on a head mounted display (HMD) for a user. Then, determining a game intensity value for said game state based on user intensity feedback received from one or more prior gaming sessions by users. The user intensity feedback describes levels of sensory stimulus obtained from the users during user play of the game via an HMD of the respective users, and the determined game intensity value is calculated by analysis of the levels of sensory stimulus received from the users. Processing the game intensity value to cause a higher rendering priority to a viewpoint of the user's gaze than regions outside of the viewpoint. The viewpoint is a portion of the display of the HMD. The regions outside of the viewport are caused to include a reduction in interactive content presented during execution of the current session of the game.

Sensory stimulus management in head mounted display

Systems and methods for executing content to be rendered on a screen of a head mounted display (HMD) are provided. One method includes executing the content to render interactive scenes on the screen and tracking an orientation direction of the HMD when worn on a head of a user and the interactive scenes are being rendered on the screen. The method includes changing view directions into the interactive scenes based on changes in the orientation direction of the HMD, such that movements of the head of the user causes the changes in the view directions into the interactive scenes. The method further includes receiving images of a real world space using a camera of the HMD. The camera of the HMD is configured to capture a location of real world objects in the real world space relative to the user of the HMD. The method includes detecting that at least one real world object is becoming proximate to the user of the HMD and generating a warning or message to be presented to the HMD, the warning or message indicating that the user is likely to bump or contact the at least one real world object. The method further includes transitioning at least a portion of the screen to a transparent mode. The transparent mode provides at least a partial view into the real world space using the camera of the HMD.

Head mounted displays (HMDs) with front facing cameras for transitioning between non-transparent modes and transparent modes

A device for interfacing with an interactive program is provided, including: a housing, including, an IR projector for projecting an IR pattern into an interactive environment; a camera for capturing images of the projected IR pattern; an image processor configured to process the captured images for identification of the projected IR pattern; a transceiver configured to communicate with a computing device, the transceiver configured to transmit data generated by the image processor to the computing device, such that a position of the device in the interactive environment is determined based on the captured images of the IR pattern, the determined position of the device being utilized as input for the interactive program.

Device for interfacing with a computing program using a projected pattern

A device for interfacing with an interactive program, including: a housing, including, an IR projector for projecting an IR pattern into an interactive environment; a camera for capturing images of the projected IR pattern; an image processor configured to process the captured images for identification of the projected IR pattern; a transceiver configured to communicate with a computing device that executes the interactive program, the transceiver configured to transmit data generated by the image processor to the computing device, such that a position of the device in the interactive environment is determined based on the captured images of the IR pattern, the determined position of the device being utilized to update a state of the executing interactive program at the computing device.

A system and method for operating a head mounted display (HMD) is provided. The HMD has a head attachment portion and a viewing module coupled to the head attachment portion. The viewing module includes an inner side having a view port into a screen configuring for rendering a virtual reality scene and an exterior housing. A communications module for exchanging data with a computer system is provided and the computer system is configured to generate the virtual reality scene for the screen. A depth camera is integrated into the viewing module and is oriented to capture depth data of an environment in front of the exterior housing. One method includes processing, by the computer system, the depth data captured by the depth camera to identify hands of a user wearing the HMD in the environment. The hands are rendered into the virtual reality scene. The hands are tracked such that movements of the hands appear as movements of virtual hands extending into the virtual reality scene.

Head mounted display

Methods, systems, and computer programs are presented for managing the sensory stimulus generated by a head mounted display (HMD). One method includes an operation for identifying the game state of a game being executed for display on a head mounted display (HMD). Further, the method includes an operation for determining the game intensity value for the game state based on user intensity ratings obtained from user play of the game. The user intensity ratings represent the level of sensory stimulus presented to users during the user play of the game via the HMDs of the respective users. Further, the method includes an operation for presenting the game intensity value to the HMD for rendering during execution of the game.

Consumer electronic devices have been developed with enormous information processing capabilities, high quality audio and video outputs, large amounts of memory, and may also include wired and/or wireless networking capabilities. Additionally, relatively unsophisticated and inexpensive sensors, such as microphones, video camera, GPS or other position sensors, when coupled with devices having these enhanced capabilities, can be used to detect subtle features about users and their environments. A variety of audio, video, simulation and user interface paradigms have been developed to utilize the enhanced capabilities of these devices. These paradigms can be used separately or together in any combination. One paradigm automatically creating user identities using speaker identification. Another paradigm includes a control button with 3-axis pressure sensitivity for use with game controllers and other input devices.

Audio, video, simulation, and user interface paradigms

A controller () for controlling an execution of a game program by a processor for enabling an interactive game to be played by a user includes a body () having a section to be oriented towards a screen when a progress of a game provided via execution of the game apparatus is displayed upon the screen, and at least one photonically detectable ('PD') element (e.g., and/or) assembled with the body, a position of the photonically detectable element within an image being recordable by an image capture device () when the section is oriented at least partly towards the screen, wherein positions of the PD element at different points in time are quantifiable to quantify movement of the body in space.

Method and apparatus for tracking three-dimensional movements of an object using a depth sensing camera

A method for executing a game presented on a screen of a head mounted display (HMD) is provided. Execution renders interactive scenes of the game on the screen. The method includes receiving data of the HMD worn by a user to identify an HMD spatial position. The method also includes receiving data of a controller held by the user to identify a controller spatial position. During game play, the method includes rendering a point of view into the interactive scenes of the game on the screen, where the point of view is a volume of space rendered in three-dimensions. The method includes adjusting the point of view as the HMD spatial position is detected to change. The controller can drive interactivity in the game and changes in position of the controller and the HMD are correlated to provide an immersive three-dimensional game play experience.

Methods for rendering interactive content to a head mounted display

A device for enabling user control for interfacing with a video game is provided, including the following: a projector for projecting a radiation pattern; a camera for capturing images of the projected radiation pattern; an image processor configured to process the captured images for identification of the projected radiation pattern; a transceiver configured to communicate with a game console that executes the video game, the transceiver configured to transmit data generated by the image processor to the game console, such that a position of the device is determined based on the captured images of the radiation pattern, the determined position of the device being utilized to update a state of the executing video game at the game console; a harness for attaching the device to the user.

A tracking device for use in obtaining information for controlling an execution of a game program by a processor for enabling an interactive game to be played by a user and related apparatus are disclosed.

Growing community of inventors

Foster City, CA, United States of America

Xiaodong Mao