Facebook, Inc.

Nvidia Corporation

Meta Platforms Technologies, LLC

Silicon Graphics, Incorporated

Ailive, Inc.

Microsoft Technology Licensing, LLC

Facebook Technologies, LLC

Lytro, Inc.

Brian Keith Cabral

Forrest Samuel Briggs

Michael John Toksvig

Joyce Hsu

Albert Parra Pozo

Rui Manuel Bastos

Andrew Hamilton Coward

Xiaoyuan Tu

Wei Yen

Karim Abdalla

John Funge

Ian Wright

Roger L Allen

Daniel Dobson

Nigel Duffy

Sohei Takemoto

Michael David McNally

Hu He

Ron Musick

Philip Nemec

Ricardo Jansson Motta

Edward Anthony Hutchins

Michael J M Toksvig

Mark S Peercy

Peter Vajda

John Milligan Airey

Elena Ing

Marc Olano

Michael F Cohen

Brett Coon

Emmett Michael Kilgariff

John R Nickolls

Sean Jeffrey Treichler

Colvin Pitts

Yi-Ren Ng

Chia-Kai Liang

Kurt Akeley

Ganesh Venkatesh

Christian Rouet

Timothy James Knight

Gary M Tarolli

Samuel Alan Johnson

Craig Michael Wittenbrink

Johnny S Rhoades

Robert C Keller

Mateusz Berezecki

John Douglas Tynefield, Jr

Michael Alexander Snower

Amit Kumar

Bennett Wilburn

Rakesh Ranjan

Joseph Virskus

Kai Li

Shen-Chi Chen

Wei Ye

Yash Patel

Mitchell Harwell

Amy J Migdal

Jonathan J Huang

Andrew Garrod Bosworth

Shang-Hung Lin

Michael Joseph Hopcroft

Christopher D S Donham

Ross Cunniff

Ignatius B Tjandrasuwita

Andrea Colaco

Gary C King

Sean Midthun Pieper

Timo Juhani Ahonen

James L Foran

Amnon Silverstein

Ronit Kassis

Freddy Abnousi

Christopher Donham

Ryan Cairns

Myriam Amsallem

Aleksandra Pang

Isaac Robert Galatzer-Levy

Elena Lamburn

Daniel Hugenroth

Ibrahim Okuyucu

Michael Jm Toksvig

Amy J Midgal

Karim M Adballa

Karim M Abdalla

Johnny B Rhoades

Systems and methods are provided for presenting a qualitative descriptor of a user's physiological state at a wrist-wearable device. The method includes monitoring, via one or more sensors, values for a plurality of physiological parameters for a user wearing the wrist-wearable device. The method includes comparing the values for the plurality of physiological parameters to baseline values for the physiological parameters. The baseline values are determined based on values for the plurality of physiological parameters that were measured over a predetermined period of time. The method includes, based on the comparison, determining a qualitative descriptor of the user's physiological state from among a set of three or more predefined qualitative descriptors. The method includes presenting, on a display that is in communication with the wrist-wearable device, the qualitative descriptor of the user's physiological state without displaying a numeric score representing the user's physiological state.

Wearable devices for providing a qualitative and actionable descriptor of a user's physiological state, and methods of use thereof

A 3D conversation system can facilitate 3D conversations in an augmented reality environment, allowing conversation participants to appear as if they are face-to-face. The 3D conversation system can accomplish this with a pipeline of data processing stages, which can include calibrate, capture, tag and filter, compress, decompress, reconstruct, render, and display stages. Generally, the pipeline can capture images of the sending user, create intermediate representations, transform the representations to convert from the orientation the images were taken from to a viewpoint of the receiving user, and output images of the sending user, from the viewpoint of the receiving user, in synchronization with audio captured from the sending user. Such a 3D conversation can take place between two or more sender/receiving systems and, in some implementations can be mediated by one or more server systems. In various configurations, stages of the pipeline can be customized based on a conversation context.

3D conversations in an artificial reality environment

A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.

Holographic calling for artificial reality

3d conversations in an artificial reality environment

In one embodiment, a computing system may receive, from a second computing system, video streams of a scene, the video streams including at least a first image and a second image that are simultaneously captured by a first camera and a second camera of the second computing system, respectively. The system may determine, using a sensor system, a viewpoint of a viewer with respect to a display region of a monoscopic display associated with the first computing system. The system may generate an output image of the scene by blending, according to blending proportions computed using the viewpoint of the viewer, corresponding portions of the first image and the second image. The system may display the output image in the display region of the monoscopic display.

Videotelephony with parallax effect

A method for using a virtual camera location to display image data to one device from another device associated with a plurality of cameras. The method includes receiving image data from a plurality of cameras associated with a second user device associated with a second user. The method further includes establishing a virtual camera location different from positions of the plurality of cameras. The method further includes providing an image using the image data on a first user display of a first user device based at least on a viewpoint of the virtual camera location.

Systems and methods for video communication using a virtual camera

In one embodiment, a method includes a computing system which may receive captured image data from a camera assembly. The camera assembly may include one or more cameras and an actuator to move the cameras. The image data may include frames that were captured while the actuator was moving the cameras. The computing system may determine for each frame, the time and position of the camera at the point that the image data was captured. The computing system may then generate an index of the captured image data and the associated time and position data. The index may be utilized for reconstructing the scene around the camera assembly from a viewpoint which is different from the viewpoint of any one of the cameras. This reconstruction may be performed based on one or more frames of the captured image data queried using the index, the viewpoint, and a time of interest.

Systems and methods for capturing image data for recreation in a virtual environment

A camera system captures images from a set of cameras to generate binocular panoramic views of an environment. The cameras are oriented in the camera system to maximize the minimum number of cameras viewing a set of randomized test points. To calibrate the system, matching features between images are identified and used to estimate three-dimensional points external to the camera system. Calibration parameters are modified to improve the three-dimensional point estimates. When images are captured, a pipeline generates a depth map for each camera using reprojected views from adjacent cameras and an image pyramid that includes individual pixel depth refinement and filtering between levels of the pyramid. The images may be used generate views of the environment from different perspectives (relative to the image capture location) by generating depth surfaces corresponding to the depth maps and blending the depth surfaces.

Panoramic camera systems

A camera system provides a live preview that provides a user device a pseudo-real time depiction of what the camera assembly is imaging. The camera system captures images from a plurality of cameras. The camera system captures images from a plurality of cameras. The camera system stores the captured images in respective memory locations of a buffer. The stored captured images form a high priority data stream that generates content associated with the portion of the local area. The camera system selects, as part of a low priority data stream, one or more of the images from memory locations. The camera system encodes the selected one or more images. The camera system packetizes the encoded one or more images to form an image frame in a video feed. The camera system provides the image frame to a user device that presents the image frame as part of the video feed.

Three-dimensional, 360-degree virtual reality camera live preview

A camera calibration system jointly calibrates multiple cameras in a camera rig system. The camera calibration system obtains configuration information about the multiple cameras in the camera rig system, such as position and orientation for each camera relative to other cameras. The camera calibration system estimates calibration parameters (e.g., rotation and translation) for the multiple cameras based on the obtained configuration information. The camera calibration system receives 2D images of a test object captured by the multiple cameras and obtains known information about the test object such as location, size, texture and detailed information of visually distinguishable points of the test object. The camera calibration system then generates a 3D model of the test object based on the received 2D images and the estimated calibration parameters. The generated 3D model is evaluated in comparison with the actual test object to determine a calibration error. The calibration parameters for the cameras are updated to reduce the calibration error for the multiple cameras.

Camera calibration system

A camera system captures images from a set of cameras to generate binocular panoramic views of an environment. The cameras are oriented in the camera system to maximize the minimum number of cameras viewing a set of randomized test points. To calibrate the system, matching features between images are identified and used to estimate three-dimensional points external to the camera system. Calibration parameters are modified to improve the three-dimensional point estimates. When images are captured, a pipeline generates a depth map for each camera using reprojected views from adjacent cameras and an image pyramid that includes individual pixel depth refinement and filtering between levels of the pyramid. The images may be used to generate views of the environment from different perspectives (relative to the image capture location) by generating depth surfaces corresponding to the depth maps and blending the depth surfaces.

Generating depth maps for panoramic camera systems

Growing community of inventors

San Jose, CA, United States of America

Brian Keith Cabral