Sony Interactive Entertainment Inc.

Sony Corporation

Yutaka Yokokawa

Nicolas Doucet

Keiichiro Ishihara

Takumi Yoshida

Yuki Miyamae

Masafumi Noda

Gento Morita

Kazutomo Sasaki

Ngo Tuyen Thibaud

Jeffrey Roger Stafford

Tetsugo Inada

Jorge Arroyo Palacios

Xiaoyong Ye

Tuyen Thibaud Ngo

Methods and apparatus provide for carrying out a method for rendering a virtual-three dimensional space on a display, including: acquiring information indicative of a position of a head mounted display worn on a head of a user; acquiring information indicative of a position of an input apparatus for use by the user; acquiring information indicative of an instruction entered in the input apparatus; and generating an image of the virtual three-dimensional space to be displayed on the head mounted display at a viewpoint position and a sightline direction, wherein the generating includes generating an image of the input apparatus at the position in the image determined on a basis of a relative position between the head mounted display of the input apparatus, and wherein the generating includes generating an image of an object which is emitted from the input apparatus in response to the user manipulating the input apparatus.

Display control program, display control apparatus and display control method

A display control apparatus includes: a head mounted display information acquisition block configured to acquire information indicative of a position of a head mounted display worn on a head of a user; an input apparatus information acquisition block configured to acquire information indicative of a position of an input apparatus for use by the user; an instruction input acquisition block configured to acquire information indicative of an instruction entered in the input apparatus; and an image generation block configured to generate an image to be displayed on the head mounted display on a basis of an instruction acquired by the instruction input acquisition block. The image generation block displays an image of the input apparatus at a position in the image determined on a basis of a relative position with the head mounted display of the input apparatus.

Display control apparatus, display control method, and recording medium

Methods and apparatus provide for carrying out a method for rendering a virtual-three dimensional space on a display, including: acquiring information indicative of a position of a head mounted display worn on a head of a user; acquiring information indicative of a position of an input apparatus for use by the user; acquiring information indicative of an instruction entered in the input apparatus; and generating an image of the virtual three-dimensional space to be displayed on the head mounted display at a viewpoint position and a sightline direction, where the generating includes generating an image of the input apparatus at the position in the image determined on a basis of a relative position with the head mounted display of the input apparatus, and where the generating includes generating an image of a hand of the user when the user controls the input apparatus.

A method for dynamic adjustment of interactive game play includes identifying a game session for a game played between first and second players, with the first and second players being connected from geographic locations having first and second elevations, respectively. The method also includes determining an objective in the game that the first and second players are predicted to achieve, and identifying a first path to be traversed by the first player to reach the objective in the game and identifying a second path to be traversed by the second player to reach the objective in the game. Each of the first and second paths includes a respective plurality of game actions to be accomplished by the first and second players. The method further includes adjusting a physical activity rating of select ones of the respective plurality of game actions based on a difference between the first and second elevations.

Altitude-based difficulty adjustments for multi-player gaming interactivity

To facilitate control of an AR HMD, a camera unit in a camera sensor system generates RGB/IR images and the system also extrapolates images for times in the future based on light intensity change signals from an event detection sensor (EDS) for HMD pose tracking, hand tracking, and eye tracking. The times in the future are provided by an HMD application which defines the future times, and the RGB/IR images and extrapolated images are sent back to the application. In this way, the camera sensor system enables improved performance tracking (equivalent to using very high-speed camera) at lower bandwidth and power consumption.

Combined light intensity based CMOS and event detection sensor for high speed predictive tracking and latency compensation in virtual and augmented reality HMD systems

Display control apparatus, display control method, and recording medium for setting viewpoint and sightline in a virtual three-dimensional space

An image from a camera on, e.g., a VR HMD is identified of a hand grasping a computer game controller. The image may be cropped to a region containing the controller and the hand to simplify processing, and a virtual representation of the hand is presented on a display such as the HMD, with the virtual representation being generated based on both image analysis of the region and a touch signal from the controller.

Image based finger tracking plus controller tracking

A 3D object is opened for presentation on a display in response to a trigger, such as a gaze direction detected by eye tracking in conjunction with a trigger hand gesture. A player's emulated hand in emulated space is configured to have the same gesture as the player's real hand as imaged by a camera, and only when the emulated hand is within the 3D object in emulated space are gestures of the hand correlated to input commands. Otherwise, hand gestures are not considered for correlation to commands.

Gesture-based user interface for AR and VR with gaze trigger

To improve the social interaction and interaction with external objects for a user wearing an AR head-mounted display (HMD), cameras on the HMD detect that other users ('social targets') are looking at and/or speaking to the user. The region around the social target in the AR user's view is determined and in response an optical occlusion system in the HMD is changed to allow more outside light to reveal the HMD user's eyes to the other users, The HMD optionally decreases the brightness of the virtual images presented in the HMD to allow the HMD user to improve his real world social and scene interaction.

Social and scene target awareness and adaptation of an occlusion system for increased social and scene interaction in an optical see-through augmented reality head mounted display

Methods and apparatus for controlling a display include: acquiring information indicative of first entered instructions from a first input apparatus used by a first user wearing a head mounted display and information indicative of second entered instructions from a second input apparatus used by a second user enabled to view a display apparatus; controlling, in a virtual three-dimensional space, an event on the basis of one or more of the first entered instructions and the second entered instructions acquired; generating a first image of said virtual three-dimensional space to be displayed on said head mounted display; and generating a second image of said virtual three-dimensional space to be displayed on said display apparatus, where the first image includes the first input apparatus, and where the second image does not include the second input apparatus.

A mat has an embedded array of light sensors under a micro-faceted lens array to provide an array of light intensity values for various incident angles of incident real world light. Virtual objects may be projected onto the mat and, using the information from the light sensors, virtually illuminated to match the real-world lighting environment. A touch sensor array may be embedded in the mat to allow users to interact with the virtual objects and provide other input. The surface of the micro-faceted lens array may be coated with a patterned infrared (IR) reflective coating, such that the mat passively reflects a specific band of IR wavelengths in a pattern to act as a fiducial marker.

Augmented reality (AR) mat with light, touch sensing mat with infrared trackable surface

A mobile device such as a mobile phone is used to image a nearby user for eye tracking at a closer range than would be provided by cameras on a display device such as a TV and send the eye tracking data to a server streaming video being presented on the display. The eye tracking data is used by the server to identify a region of interest (ROI) in the video, which is more highly compressed than regions outside the ROI to facilitate efficient compression of the video game image while preserving the quality of the ROI in the image where the user is looking.

Foveated optimization of TV streaming and rendering content assisted by personal devices

Disclosed herein is an information processing apparatus, including: an image acceptance section configured to accept an image of an operation inputting apparatus picked up by an imaging section; an operation signal acceptance section configured to accept an operation signal representative of an operation for the operation inputting apparatus from the operation inputting apparatus; and a process execution section configured to execute a process in response to the position of the operation inputting apparatus in the image accepted by the image acceptance section and the operation signal accepted by the operation signal acceptance section.

Information processing apparatus, information processing system, information processing method, program and information recording medium

A method is provided, including the following operations: receiving, from a controller device, controller input that identifies postures of at least two fingers of the user's hand; determining a similarity of the controller input to a predefined target input; rendering in a virtual space a virtual hand that corresponds to the controller device, wherein when the similarity exceeds a predefined threshold, then, in response the virtual hand is animated so that a pose of the virtual hand transitions to a predefined hand pose, such that postures of fingers of the virtual hand transition to predefined finger postures of the predefined hand pose, and wherein when the similarity does not exceed the predefined threshold, then the virtual hand is rendered so that the pose of the virtual hand dynamically changes in response to changes in the controller input.

Growing community of inventors

Kanagawa, Japan

Yutaka Yokokawa