Magic Leap, Inc.

Salesforce Com, Inc.

The Climate Corporation

Salesforce, Inc.

Ankur Gupta

Mohamed Souiai

Anuj Gargeya Malkapuram

Ying Xu

Lijuan Xu

Moshe Bouhnik

Igor Napolskikh

Yu-Hsiang Huang

Vikram Chauhan

Konstantinos Zampogiannis

Etienne Gregoire Grossman

Changxin Yan

Nick Donaldson

David Gadot Kabasu

Gil Sokol

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing miscalibration detection. One of the methods includes receiving sensor data from each of multiple sensors of a device in a system configured to provide augmented reality or mixed reality output to a user. Feature values are determined based on the sensor data for a predetermined set of features. The determined feature values are processed using a miscalibration detection model that has been trained, based on examples of captured sensor data from one or more devices, to predict whether a miscalibration condition of one or more of the multiple sensors has occurred. Based on the output of the miscalibration detection model, the system determines whether to initiate recalibration of extrinsic parameters for at least one of the multiple sensors or to bypass recalibration.

Miscalibration detection for virtual reality and augmented reality systems

Techniques for addressing deformations in a virtual or augmented headset described. In some implementations, cameras in a headset can obtain image data at different times as the headset moves through a series of poses of the headset. One or more miscalibration conditions for the headset that have occurred as the headset moved through the series of poses can be detected. The series of poses can be divided into groups of poses based on the one or more miscalibration conditions, and bundle adjustment for the groups of poses can be performed using a separate set of camera calibration data. The bundle adjustment for the poses in each group is performed using a same set of calibration data for the group. The camera calibration data for each group is estimated jointly with bundle adjustment estimation for the poses in the group.

Calibration for virtual or augmented reality systems

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calibrating an augmented reality device using camera and inertial measurement unit data. In some implementations, a bundle adjustment process jointly optimizes or estimates states of the augmented reality device. The process can use, as input, visual and inertial measurements as well as factory-calibrated sensor extrinsic parameters. The process performs bundle adjustment and uses non-linear optimization of estimated states constrained by the measurements and the factory calibrated extrinsic parameters. The process can jointly optimize inertial constraints, IMU calibration, and camera calibrations. Output of the process can include most likely estimated states, such as data for a 3D map of an environment, a trajectory of the device, and/or updated extrinsic parameters of the visual and inertial sensors (e.g., cameras and IMUs).

Joint camera and inertial measurement unit calibration

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using depth data to update camera calibration data. In some implementations, a frame of data is captured including (i) depth data from a depth sensor of a device, and (ii) image data from a camera of the device. Selected points from the depth data are transformed, using camera calibration data for the camera, to a three-dimensional space that is based on the image data. The transformed points are projected onto the two-dimensional image data from the camera. Updated camera calibration data is generated based on differences between (i) the locations of the projected points and (ii) locations that features representing the selected points appear in the two-dimensional image data from the camera. The updated camera calibration data can be used in a simultaneous localization and mapping process.

Camera calibration using depth sensor data

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for camera calibration during bundle adjustment. One of the methods includes maintaining a three-dimensional model of an environment and a plurality of image data clusters that each include data generated from images captured by two or more cameras included in a device. The method includes jointly determining, for a three-dimensional point represented by an image data cluster (i) the newly estimated coordinates for the three-dimensional point for an update to the three-dimensional model or a trajectory of the device, and (ii) the newly estimated calibration data that represents the spatial relationship between the two or more cameras.

Concurrent camera calibration and bundle adjustment

Devices are described for high accuracy displacement of tools. In particular, embodiments provide a device for adjusting a position of a tool. The device includes a threaded shaft having a first end and a second end and a shaft axis extending from the first end to the second end, a motor that actuates the threaded shaft to move in a direction of the shaft axis. In some examples, the motor is operatively coupled to the threaded shaft. The device includes a carriage coupled to the camera, and a bearing assembly coupled to the threaded shaft and the carriage. In some examples, the bearing assembly permits a movement of the carriage with respect to the threaded shaft. The movement of the carriage allows the position of the camera to be adjusted.

High accuracy displacement device

An online system monitors resources utilization by users connecting with the online system and detects unauthorized resource utilization caused by sharing of sessions. The online system collects samples of browser attributes from browsers interacting with the online system. The online system determines a score indicating a difference between two samples of browser attributes taken at different times. The online system uses the score to determine whether the two samples of browser attributes in the same session were received from different browsers. If the online system detects unauthorized resource utilization if the two samples are determined to be from two different browsers. The online system takes mitigating actions, for example, by invalidating the session or requiring users to re-enter credentials.

Monitoring resource utilization of an online system based on browser attributes collected for a session

An encoder receives an application log file including component values and encodes the component values into lists of preliminary encoded values. The lists of preliminary encoded values are combined into a combined list of preliminary encoded values. An encoder-decoder neural network is trained to encode the combined list of preliminary encoded values into a list of collectively encoded values, to decode the list of collectively encoded values into a list of decoded values, and to optimize a metric measuring the encoder-decoder neural network's functioning, in response to receiving the combined list of preliminary encoded values. The trained encoder-decoder neural network receives combined lists of preliminary encoded values for application log files and encodes the combined lists of preliminary encoded values into lists of collectively encoded values. The lists of collectively encoded values are sent to a detector, thereby enabling the detector to detect an application event associated with the application log files.

Detecting application events based on encoding application log values

A computer implemented method for generating digital models of relative crop yield based on nitrate values in the soil is provided. In an embodiment, nitrate measurements from soil during a particular portion of a crop's development and corresponding crop yields are received by an agricultural intelligence computing system. Based, at least in part, on the nitrate measurements and corresponding crop yields, the system determines maximum yields for each location of a plurality of locations. The system then converts each crop yield value into a relative crop yield by dividing the crop yield value by the maximum crop yield for the location. Using the relative crop yields and the corresponding nitrate values in the soil, the system generates a digital model of relative crop yield as a function of nitrate in the soil during the particular portion of the crop's development. When the system receives nitrate measurements from soil in a particular field during the particular portion of a crop's development, the system computes a relative yield value for the particular field using the model of relative crop yield.

Generating digital models of relative yield of a crop based on nitrate values in the soil

An online system monitors resources utilization by users connecting with the online system and detects unauthorized resource utilization. The online system collects samples of browser attributes from browsers interacting with the online system. The online system determines statistics describing the browser attributes based on the collected samples for that user. The online system receives values of browser attributes for a new request received from a user and determines a browser score indicating a likelihood that the new request was sent from a new client device different from the client devices used by the user during the time interval. If the online system determines that the score indicates that the new request was sent by the new client device, the online system takes mitigating actions to control the unauthorized resource utilization, for example, by requesting credentials for authenticating the request.

Growing community of inventors

San Francisco, CA, United States of America

Ankur Gupta