Meta Platforms Technologies, LLC

Microsoft Technology Licensing, LLC

Facebook Technologies, LLC

Chao Li

Richard Andrew Newcombe

Andrew Samuel Berkovich

Steven John Lovegrove

Lingni Ma

Li Erran Li

Ankit Saraf

Weizhu Chen

Eslam K Abdelreheem

Jiayuan Huang

Daniel Andersen

Changhong Yuan

Michael Goesele

Grace Elizabeth Kuo

Hongjun Qiang

Xiaoying Guo

Yan Jiang

Yunjing Ma

Shean Wang

Tsuyoshi Watanabe

Yang Shu

Jaclyn Ruth Elizabeth Phillips

Yuantao Wang

Qi Ren

Justin Carl Wong

Nan Zhao

Erlei Xu

Yujia Chen

Niangjun Zhu

Zheng Shi

In particular embodiments, a computing system may capture a first image of a scene using a first camera of an artificial reality device. The system may capture a second image of the scene using a second camera and one or more optical elements of the artificial reality device. The second image may include an overlapping portion of multiple shifted copies of the scene. The system may generate an upsampled first image by applying a particular sampling technique to the first image. The system may generate a tiled image comprising a plurality of repeated second images by applying a tiling process to the second image. The system may generate an initial output image by processing the upsampled first image and the tiled image using a machine learning model. The system may generate a final output image by normalizing the initial output image using the upsampled first image.

Generating super resolution images using duo-camera artificial reality device

In some examples, iterative sampling based dataset clustering may include sampling a dataset that includes a plurality of items to identify a specified number of sampled items. The sampled items may be clustered to generate a plurality of clusters. Un-sampled items may be assigned from the plurality of items to the clusters. Remaining un-sampled items that are not assigned to the clusters may be identified. A ratio associated with the remaining un-sampled items and the plurality of items may be compared to a specified threshold. Based on a determination that the ratio is greater than the specified threshold, an indication of completion of clustering of the plurality of items may be generated.

Iterative sampling based dataset clustering

In one embodiment, a computing system instructs, at a first time, a camera having a plurality of pixel sensors to use the plurality of pixel sensors to capture a first image of an environment comprising an object. The computing system predicts, using at least the first image, a projection of the object appearing in a virtual image plane associated with a predicted camera pose at a second time. The computing system determines, based on the predicted projection of the object, a first region of pixels and a second region of pixels. The computing system generates pixel-activation instructions for the first region of pixels and the second region of pixels. The computing system instructs the camera to capture a second image of the environment at the second time according to the pixel-activation instructions. The pixel-activation instructions are configured to cause a first subset of the plurality of pixel sensors to sample the first region of pixels and a second subset of the plurality of pixel sensors to sample the second region of pixels. The first subset of the plurality of pixel sensors used for sampling the first region of pixels is more dense than the second subset of the plurality of pixel sensors used for sampling the second region of pixels. The computing system tracks, based on the second image, the object at the second time.

Object tracking using sparse sensor captures

In one embodiment, a method includes instructing, at a first time, a camera having a plurality of pixel sensors to capture a first image of an environment comprising an object to determine a first object pose; determining, based on the first object pose, a predicted object pose of the object at a second time; generating pixel-activation instructions based on a buffer region around a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with a predicted camera pose, where the size of the buffer region may be dependent on predicted dynamics for the object; instructing, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions, and; determining, based on the second image, a second object pose of the object.

Techniques for remote control of applications are described. A method according to an aspect of the disclosure comprises receiving, from an input processing unit, one ore more commands for performing one or more operations of an application instance; identifying, through checking pairing information, the application instance paired with the input processing unit; and sending to the identified application instance, the received one or more commands.

Remote control of applications

In one embodiment, a method includes instructing, at a first time, a camera with multiple pixel sensors to capture a first image of an environment comprising an object to determine a first object pose of the object. Based on the first object pose, the method determines a predicted object pose of the object at a second time. The method determines a predicted camera pose of the camera at the second time. The method generates pixel-activation instructions based on a projection of a 3D model of the object having the predicted object pose onto a virtual image plane associated with the predicted camera pose. The method instructs, at the second time, the camera to use a subset of the plurality of pixel sensors to capture a second image of the environment according to the pixel-activation instructions. The method determines, based on the second image, a second object pose of the object.

Growing community of inventors

Redmond, WA, United States of America

Chao Li