Facebook, Inc.

Nvidia Corporation

Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Oculus, Vr, LLC

Meta Platforms, Inc.

Michael John Toksvig

Brian Keith Cabral

Forrest Samuel Briggs

David Harry Garcia

Shafigh Shirinfar

Joyce Hsu

Dov Katz

Torin Ross Herndon

Ziheng Wang

Timothy Paul Omernick

Yoon Kean Wong

Jonah Matthew Alben

Walter E Donovan

Stephen Donald Lew

Joshua Wiseman

Krishnaraj S Rao

Eglia Nair Flores

Erik Lindholm

Erick Tseng

Matthew Nicholas Papakipos

Yael Gregory Maguire

Emmett Michael Kilgariff

Andrew Garrod Bosworth

Sean Jeffrey Treichler

Rui Manuel Bastos

Mohit Talwar

Christian Rouet

Roger L Allen

Gary M Tarolli

Charles Joseph Hughes

Craig Michael Wittenbrink

Adrian Potra

Andrew Hamilton Coward

Patrick Brendan Maginnis

John Douglas Tynefield, Jr

Freddy Abnousi

Karim M Abdalla

Myriam Amsallem

Aleksandra Pang

Isaac Robert Galatzer-Levy

Craig Michael Nicol

Johnny B Rhoades

Systems and methods are provided for presenting a qualitative descriptor of a user's physiological state at a wrist-wearable device. The method includes monitoring, via one or more sensors, values for a plurality of physiological parameters for a user wearing the wrist-wearable device. The method includes comparing the values for the plurality of physiological parameters to baseline values for the physiological parameters. The baseline values are determined based on values for the plurality of physiological parameters that were measured over a predetermined period of time. The method includes, based on the comparison, determining a qualitative descriptor of the user's physiological state from among a set of three or more predefined qualitative descriptors. The method includes presenting, on a display that is in communication with the wrist-wearable device, the qualitative descriptor of the user's physiological state without displaying a numeric score representing the user's physiological state.

Wearable devices for providing a qualitative and actionable descriptor of a user's physiological state, and methods of use thereof

In one embodiment, a method includes a computing system which may receive captured image data from a camera assembly. The camera assembly may include one or more cameras and an actuator to move the cameras. The image data may include frames that were captured while the actuator was moving the cameras. The computing system may determine for each frame, the time and position of the camera at the point that the image data was captured. The computing system may then generate an index of the captured image data and the associated time and position data. The index may be utilized for reconstructing the scene around the camera assembly from a viewpoint which is different from the viewpoint of any one of the cameras. This reconstruction may be performed based on one or more frames of the captured image data queried using the index, the viewpoint, and a time of interest.

Systems and methods for capturing image data for recreation in a virtual environment

A virtual reality system includes a head-mounted display (HMD) having one or more facial sensors and illumination sources mounted to a surface of the HMD. For example, the facial sensors are image capture devices coupled to a bottom side of the HMD. The illumination sources illuminate portions of a user's face outside of the HMD, while the facial sensors capture images of the illuminated portions of the user's face. A controller receives the captured images and generates a representation of the portions of the user's face by identifying landmarks of the user's face in the captured images and performing other suitable image processing methods. Based on the representation, the controller or another component of the virtual reality system generates content for presentation to the user.

Tracking portions of a user's face uncovered by a head mounted display worn by the user

A camera system captures images from a set of cameras to generate binocular panoramic views of an environment. The cameras are oriented in the camera system to maximize the minimum number of cameras viewing a set of randomized test points. To calibrate the system, matching features between images are identified and used to estimate three-dimensional points external to the camera system. Calibration parameters are modified to improve the three-dimensional point estimates. When images are captured, a pipeline generates a depth map for each camera using reprojected views from adjacent cameras and an image pyramid that includes individual pixel depth refinement and filtering between levels of the pyramid. The images may be used generate views of the environment from different perspectives (relative to the image capture location) by generating depth surfaces corresponding to the depth maps and blending the depth surfaces.

Panoramic camera systems

As user device can receive and display 360 panoramic content in a 360 depth format. 360 depth content can comprise 360 panoramic image data and corresponding depth information. To display 360 depth content, the user device can generate a 3D environment based on the 360 depth content and the current user viewpoint. A content display module on the user device can render 360 depth content using a standard 3D rendering pipeline modified to render 360 depth content. The content display module can use a vertex shader or fragment shader of the 3D rendering pipeline to interpret the depth information of the 360 depth content into the 3D environment as it is rendered.

Rendering 360 depth content

A parallax viewer system allows 3D content, such as 360 degree 3D panoramas or other 3D environments, to be viewed by a user through a traditional 2D screen. A parallax viewer system operating on a user device can use a user-facing camera and a 2D screen to simulate a 3D environment for a user viewing the 2D screen. By changing the rendered view of the 3D environment as the user's head moves with respect to the screen, the parallax viewer system can provide many of the immersion benefits of a VR or 3D display using a traditional 2D display. In some implementations, the parallax viewer system can be calibrated to work in new situations (for example, on a new user device) by determining the relationship between the user-facing camera and the screen used to display the virtual environment.

Parallax viewer system calibration

Parallax viewer system for 3D content

A camera system captures images from a set of cameras to generate binocular panoramic views of an environment. The cameras are oriented in the camera system to maximize the minimum number of cameras viewing a set of randomized test points. To calibrate the system, matching features between images are identified and used to estimate three-dimensional points external to the camera system. Calibration parameters are modified to improve the three-dimensional point estimates. When images are captured, a pipeline generates a depth map for each camera using reprojected views from adjacent cameras and an image pyramid that includes individual pixel depth refinement and filtering between levels of the pyramid. The images may be used to generate views of the environment from different perspectives (relative to the image capture location) by generating depth surfaces corresponding to the depth maps and blending the depth surfaces.

Generating depth maps for panoramic camera systems

Advertisement content associated with an entity having a physical location is selected for display to a user by a client device. A proximity of the client device to the physical location is determined. In response to determining the proximity between the client device and the physical location, an advertisement is generated that includes the advertisement content and an indicator of the physical location. The generated advertisement is sent to the client device for display to the user.

Displaying advertisements with location information

An online system identifies a conversion of an advertisement when a client device associated with an online system user is within a threshold distance of a physical location associated with the advertisement. The client device sends obfuscated information identifying the user associated with the client device and the location of the client device to an independent third party when the client device is within a threshold distance of the physical location associated with the advertisement. Using information from the client device, the third party requests information from the online system identifying one or more groups including the user. Based on information identifying groups including various online system users, the third party generates information identifying conversions associated with different groups. The online system may determine effectiveness of various advertisements using the information from the third party identifying conversions associated with different groups.

Offline conversion measurement for advertisements presented to users of an online system

A 360 video system can render 360 stereoscopic content based on a virtual environment using a standard GPU rendering pipeline. In some embodiments, in order to improve efficiency in generating 360 stereoscopic content, a vertex shift technique can be used to approximate multiple viewpoints in a single 360 stereoscopic eye view. When rendering the virtual environment, each triangle of the virtual environment can be shifted to represent the view from a viewpoint corresponding to that triangle. Using vertex shift techniques, a virtual environment can be rendered into a 360 stereoscopic eye view in one pass of a GPU rendering pipeline, according to some embodiments.

Vertex shift for rendering 360 stereoscopic content

A facial tracking system generates a virtual rendering of a portion of a face of a user wearing a head-mounted display (HMD). The facial tracking system illuminates portions of the face inside the HMD. The facial tracking system captures a plurality of facial data of the portion of the face using one or more facial sensors located inside the HMD. A plurality of planar sections of the portion of the face are identified based at least in part on the plurality of facial data. The plurality of planar sections are mapped to one or more landmarks of the face. Facial animation information is generated based at least in part on the mapping, the facial animation information describing a portion of a virtual face corresponding to the portion of the user's face.

Facial animation using facial sensors within a head-mounted display

Growing community of inventors

Palo Alto, CA, United States of America

Michael John Toksvig