Qualcomm Incorporated

Tesla, Inc.

Arvind Ramanandan

Murali Chari

Christopher Brunner

Mahesh Ramachandran

Abhishek Tyagi

Andrej Karpathy

Mark Leo Moeglein

Avdhut Joshi

Paul Chen

Elon Reeve Musk

Ashok Kumar Elluswamy

Charles Shieh

Matthew Bauch

Christopher Payne

Francis Havlak

Nenad Uzunovic

Oktay Arslan

Nicklas Gustafsson

David Seelig

Christopher Coleman Moore

Kate Park

Stuart Bowers

Ashima Kapur Sud

Paril Jain

Alexander Hertzberg

Jason Kong

Dhaval Shroff

Li Wang

Hui Chao

Urs Niesen

Serafin Diaz Spindola

Paulo Ricardo Dos Santos Mendonca

James Robert Howard Hakewill

Dheeraj Ahuja

Daniel Knoblauch

Yiming Chen

John Steven Lima

Murali Ramaswamy Chari

A processor coupled to memory is configured to receive an identification of a geographical location associated with a target specified by a user remote from a vehicle. A machine learning model is utilized to generate a representation of at least a portion of an environment surrounding the vehicle using sensor data from one or more sensors of the vehicle. At least a portion of a path to a target location corresponding to the received geographical location is calculated using the generated representation of the at least portion of the environment surrounding the vehicle. At least one command is provided to automatically navigate the vehicle based on the determined path and updated sensor data from at least a portion of the one or more sensors of the vehicle.

Autonomous and user controlled vehicle summon to a target

A processor coupled to memory is configured to receive image data based on an image captured by a camera of a vehicle. The image data is used as a basis of an input to a trained machine learning model trained to predict a three-dimensional trajectory of a machine learning feature. The three-dimensional trajectory of the machine learning feature is provided for automatically controlling the vehicle.

Predicting three-dimensional features for autonomous driving

An electronic device is described. The electronic device includes a memory and a processor in communication with the memory. The memory is configured to store precalibration data for a camera mounted on a vehicle, the precalibration data including a camera height determined relative to a road plane the vehicle is configured to contact during operation. The processor is configured to receive a plurality of images. The processor is also configured to classify one or more features in the plurality of images as road features based on the precalibration data.

Systems and methods for classifying road features

Techniques provided herein are directed toward using a camera, such as a forward-facing camera, to identify non-line-of-sight (NLoS) satellites in a satellite positioning system. In particular, successive images captured by the camera of the vehicle can be used to create a three-dimensional (3-D) skyline model of one or more objects that may be obstructing the view of a satellite (from the perspective of the vehicle). Accordingly, this allows for the determination of NLoS satellites and exclusion of data from the NLoS satellites in the determination of the location of the vehicle. Techniques may further include providing the determined location of the vehicle.

Non-line-of-sight (NLoS) satellite detection at a vehicle using a camera

A mobile device determines a vision based pose using images captured by a camera and determines a sensor based pose using data from inertial sensors, such as accelerometers and gyroscopes. The vision based pose and sensor based pose are used separately in a visualization application, which displays separate graphics for the different poses. For example, the visualization application may be used to calibrate the inertial sensors, where the visualization application displays a graphic based on the vision based pose and a graphic based on the sensor based pose and prompts a user to move the mobile device in a specific direction with the displayed graphics to accelerate convergence of the calibration of the inertial sensors. Alternatively, the visualization application may be a motion based game or a photography application that displays separate graphics using the vision based pose and the sensor based pose.

Display of separate computer vision based pose and inertial sensor based pose

A method performed by an electronic device is described. The method includes determining a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU). The method also includes determining, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth. The method further includes determining a difference vector between the predicted velocity and the GPS velocity. The method additionally includes refining a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector. The method also includes refining a misalignment estimate between the camera and the IMU based on the difference vector. The method further includes providing pose information based on the refined bias estimate, the refined scale factor, and the refined misalignment estimate.

Systems and methods for using a global positioning system velocity in visual-inertial odometry

A method for visual inertial odometry (VIO)-aided global positioning is described. The method includes updating an extended Kalman filter (EKF) state including a current pose and a sliding window of multiple prior poses. The sliding window includes poses at a number of most recent global positioning system (GPS) time epochs. Updating the EKF includes updating an EKF covariance matrix for the prior poses and the current pose in the EKF state. The method also includes determining, at a GPS epoch, a relative displacement between each of the updated prior poses and the current pose. The method further includes determining an error covariance of each of the relative displacements based on cross-covariances between each of the updated prior poses and the current pose in the EKF covariance matrix. The method additionally includes using the relative displacements and the error covariances to fuse pseudorange measurements taken over multiple epochs.

Systems and methods for using a sliding window of global positioning epochs in visual-inertial odometry

A first map comprising local features and 3D locations of the local features is generated, the local features comprising visible features in a current image and a corresponding set of covisible features. A second map comprising prior features and 3D locations of the prior features may be determined, where each prior feature: was first imaged at a time prior to the first imaging of any of the local features, and lies within a threshold distance of at least one local feature. A first subset comprising previously imaged local features in the first map and a corresponding second subset of the prior features in the second map is determined by comparing the first and second maps, where each local feature in the first subset corresponds to a distinct prior feature in the second subset. A transformation mapping a subset of local features to a subset of prior features is determined.

Geometric matching in visual navigation systems

Embodiments disclosed pertain to the use of user equipment (UE) for the generation of a 3D exterior envelope of a structure based on captured images and a measurement set associated with each captured image. In some embodiments, a sequence of exterior images of a structure is captured and a corresponding measurement set comprising Inertial Measurement Unit (IMU) measurements, wireless measurements (including Global Navigation Satellite (GNSS) measurements) and/or other non-wireless sensor measurements may be obtained concurrently. A closed-loop trajectory of the UE in global coordinates may be determined and a 3D structural envelope of the structure may be obtained based on the closed loop trajectory and feature points in a subset of images selected from the sequence of exterior images of the structure.

Exterior hybrid photo mapping

An accelerometer in a mobile device is calibrated by taking multiple measurements of acceleration vectors when the mobile device is held stationary at different orientations with respect to a plane normal. A circle is calculated that fits respective tips of measured acceleration vectors in the accelerometer coordinate system. The radius of the circle and the lengths of the measured acceleration vectors are used to calculate a rotation angle for aligning the accelerometer coordinate system with the mobile device surface. A gyroscope in the mobile device is calibrated by taking multiple measurements of a rotation axis when the mobile device is rotated at different rates with respect to the rotation axis. A line is calculated that fits the measurements. The angle between the line and an axis of the gyroscope coordinate system is used to align the gyroscope coordinate system with the mobile device surface.

Method for aligning a mobile device surface with the coordinate system of a sensor

Embodiments disclosed obtain a plurality of measurement sets from a plurality of sensors in conjunction with the capture of a sequence of exterior and interior images of a structure while traversing locations in and around the structure. Each measurement set may be associated with at least one image. An external structural envelope of the structure is determined from exterior images of the structure and the corresponding outdoor trajectory of a UE. The position and orientation of the structure and the structural envelope is determined in absolute coordinates. Further, an indoor map of the structure in absolute coordinates may be obtained based on interior images of the structure, a structural envelope in absolute coordinates, and measurements associated with the indoor trajectory of the UE during traversal of the indoor area to capture the interior images.

Hybrid photo navigation and mapping

Systems, apparatus and methods for estimating gravity and/or scale in a mobile device are presented. A difference between an image-based pose and an inertia-based pose is using to update the estimations of gravity and/or scale. The image-based pose is computed from two poses and is scaled with the estimation of scale prior to the difference. The inertia-based pose is computed from accelerometer measurements, which are adjusted by the estimation for gravity.

Adaptive scale and/or gravity estimation

Growing community of inventors

San Diego, CA, United States of America

Arvind Ramanandan