Amazon Technologies, Inc.

Magic Leap, Inc.

Moshe Bouhnik

Eduard Oks

Ashwin Swaminathan

Ali Shahrokni

Matthew Alan Townsend

Miguel Andres Granados Velasquez

Ben Weisbih

Konstantin Kraimer

Hilit Unger

Noam Sorek

Ankur Gupta

Lee David Thompson

Mohamed Souiai

Konstantinos Zampogiannis

Anton Fedorenko

Siddharth Jacob Thazhathu

Nicholas Robert Ditzler

Devesh Sanghvi

David Gadot Kabasu

Gil Sokol

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing miscalibration detection. One of the methods includes receiving sensor data from each of multiple sensors of a device in a system configured to provide augmented reality or mixed reality output to a user. Feature values are determined based on the sensor data for a predetermined set of features. The determined feature values are processed using a miscalibration detection model that has been trained, based on examples of captured sensor data from one or more devices, to predict whether a miscalibration condition of one or more of the multiple sensors has occurred. Based on the output of the miscalibration detection model, the system determines whether to initiate recalibration of extrinsic parameters for at least one of the multiple sensors or to bypass recalibration.

Miscalibration detection for virtual reality and augmented reality systems

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using depth data to update camera calibration data. In some implementations, a frame of data is captured including (i) depth data from a depth sensor of a device, and (ii) image data from a camera of the device. Selected points from the depth data are transformed, using camera calibration data for the camera, to a three-dimensional space that is based on the image data. The transformed points are projected onto the two-dimensional image data from the camera. Updated camera calibration data is generated based on differences between (i) the locations of the projected points and (ii) locations that features representing the selected points appear in the two-dimensional image data from the camera. The updated camera calibration data can be used in a simultaneous localization and mapping process.

Camera calibration using depth sensor data

A portable electronic system receives a set of one or more canonical maps and determines the sparse map based at least in part upon one or more anchors pertaining to the physical environment. The sparse map is localized to at least one canonical map in the set of one or more canonical maps, and a new canonical map is created at least by merging sparse map data of the sparse map into the at least one canonical map. The set of one or more canonical maps may be determined from a universe of canonical maps comprising a plurality of canonical maps by applying a hierarchical filtering scheme to the universe. The sparse map may be localized to the at least one canonical map at least by splitting the sparse map into a plurality of connected components and by one or more merger operations.

Cross reality system with WiFi/GPS based map merge

A cross reality system receives tracking information in a tracking map and first location metadata associated with at least a portion of the tracking map. A sub-portion of a canonical map is determined based at least in part on a correspondence between the first location metadata associated with the at least the portion of the tracking map and second location metadata associated with the sub-portion of the canonical map. The sub-portion of the canonical map may be merged with the at least the portion of the tracking map into a merged map. The cross reality system may further generate the tracking map by using at least the pose information from one or more images and localize the tracking map to the canonical may at least by using a persistent coordinate frame in the canonical map and the location metadata associated with the location represented in the tracking map.

Cross reality system with WIFI/GPS based map merge

Techniques for presenting and interacting with composite images on a computing device are described. In an example, the device presents a first article and a first portion of a first composite image showing a second article. The first composite image shows a first outfit combination that is different from the first outfit. The device receives a first user interaction indicating a request to change the second article and presents the first article and a second portion of a second composite image showing a clothing article in a second outfit. The second composite image shows a second outfit combination that is different from the first outfit, the second outfit, and the first outfit combination. The device receives a second user interaction indicating a request to use the third article and presents the second composite image showing the second outfit combination.

Presenting and interacting with composite images on a computing device

A system and method for determining video clips including interesting content from video data. The system may receive annotation data identifying time and positions corresponding to objects represented in the video data and the system may determine priority metrics associated with each of the objects. By associating the priority metrics with the time and positions corresponding to the objects, the system may generate a priority metric map indicating a time and position of interesting moments in the video data. The system may generate moments and/or video clips based on the priority metric map. The system may determine a time (e.g., video frames) and/or space (e.g., pixel coordinates) associated with the moments/video clips and may simulate camera motion such as panning and/or zooming with the moments/video clips. The system may generate a Master Clip Table including the moments, video clips and/or annotation data associated with the moments/video clips.

Cataloging video and creating video summaries

A computer-implemented method includes receiving first pose data for a first human represented in a first image, receiving second pose data for a second human represented in a second image, receiving first semantic segmentation data for the first image, and receiving second semantic segmentation data for the second image. A pose-aligned second image can be generated by modifying the second image based on the first pose data, the second pose data, the first semantic segmentation data, and the second semantic segmentation data. A mixed image can be determined by combining pixel values from the first image and pixel values of the pose-aligned second image according to mask data. In some embodiments, the mixed image includes a representation of an outfit that includes first clothing represented in the first image and second clothing represented in the second image.

Method and system for generating combined images utilizing image processing of multiple images

Growing community of inventors

Holon, Israel

Moshe Bouhnik