Qualcomm Incorporated

Christopher Brunner

Murali Chari

Mahesh Ramachandran

Arvind Ramanandan

Abhishek Tyagi

Mark Leo Moeglein

Serafin Diaz Spindola

Victor Kulik

Radhika Dilip Gowaikar

Dheeraj Ahuja

Behnaz Rezaei

Stephen William Edge

Makesh Pravin John Wilson

Sven Fischer

Saumitra Mohan Das

Vinay Sridhara

Mohit Narang

Hui Chao

Wyatt Thomas Riley

Behrooz Khorashadi

Ashwin Deepak Swaminathan

Alok Aggarwal

Srinivasan Vasudevan

Volodimir Slobodyanyuk

Zoltan F Biacs

Oronzo Flore

Amin Ansari

Jayakrishnan Unnikrishnan

Michael Joshua Shomin

Avdhut Joshi

John Anthony Dougherty

Bolan Jiang

Piyush Sharma

Chiachi Sung

Paulo Ricardo Dos Santos Mendonca

Daniel Knoblauch

Jay F Dills

Jose Edson Vargas Bautista

Shantanu Chaisson Sanyal

Fu-Chun Yeh

Gary Mcgrath

Raymond A Skirsky

Murali Ramaswamy Chari

Murali R Chari

In some aspects, a device may receive sensor data associated with a vehicle and a set of frames. The device may aggregate, using a first pose, the sensor data associated with the set of frames to generate an aggregated frame, wherein the aggregated frame is associated with a set of cells. The device may obtain an indication of a respective occupancy label for each cell from the set of cells, wherein the respective occupancy label includes a first occupancy label or a second occupancy label, and wherein a subset of cells from the set of cells are associated with the first occupancy label. The device may train, using data associated with the aggregated frame, a machine learning model to generate an occupancy grid, based on a loss function that only calculates a loss for respective cells from the subset of cells. Numerous other aspects are described.

Machine learning based occupancy grid generation

Certain aspects are directed to an apparatus for lane estimation. The apparatus generally includes: at least one memory; and at least one processor coupled to the at least one memory and configured to receive a first input associated with a three-dimensional (3D) space, extract, from the first input, a first set of points associated with a ground plane of the 3D space, map each of the first set of points to a region of a plurality of regions of a two-dimensional (2D) frame, determine one or more attributes associated with each region of the plurality of regions based on one or more of the first set of points mapped to the region, and identify one or more road lanes based on the one or more attributes.

Boundary estimation

Disclosed are techniques for annotating image frames using information from a light detection and ranging (LiDAR) sensor. An exemplary method includes receiving, from the LiDAR sensor, at least one LiDAR frame, receiving, from a camera sensor, at least one image frame, removing LiDAR points that represent a ground surface of the environment, identifying LiDAR points of interest in the at least one LiDAR frame, segmenting the LiDAR points of interest to identify at least one object of interest in the at least one LiDAR frame, and annotating the at least one image frame with a three-dimensional oriented bounding box of the at least one object of interest detected in the at least one image frame by projecting the three-dimensional oriented bounding boxes from the at least one LiDAR frame to the at least one image frame using cross-calibration transforms between the LiDAR sensor and the camera.

Using light detection and ranging (LIDAR) to train camera and imaging radar deep learning networks

Embodiments include devices and methods for automatically calibrating a camera. In various embodiments, an image sensor may capture an image. Locations of one or more points including in the captured image frames may be predicted and detected. Calibration parameters may be calculated based on differences between predicted locations of a selected point within an image frame and observed locations of the selected point within the captured image frame. The automatic camera calibration method may be repeated until the calibration parameters satisfy a calibration quality threshold.

Camera auto-calibration with gyroscope

A mobile device determines a vision based pose using images captured by a camera and determines a sensor based pose using data from inertial sensors, such as accelerometers and gyroscopes. The vision based pose and sensor based pose are used separately in a visualization application, which displays separate graphics for the different poses. For example, the visualization application may be used to calibrate the inertial sensors, where the visualization application displays a graphic based on the vision based pose and a graphic based on the sensor based pose and prompts a user to move the mobile device in a specific direction with the displayed graphics to accelerate convergence of the calibration of the inertial sensors. Alternatively, the visualization application may be a motion based game or a photography application that displays separate graphics using the vision based pose and the sensor based pose.

Display of separate computer vision based pose and inertial sensor based pose

An apparatus and method for a framework exposing an API (application programming interface) to web-based server applications on the internet or in the cloud is presented. The API allows server applications to retrieve sensor data from a mobile device via a low-power sensor core processor on a mobile device. This API eliminates effort and cost associated with developing and promoting a new mobile device client application. The API framework includes APIs that web-based application may use to fetch sensor data from one or more particular sensors on the mobile device.

Sensor API framework for cloud based applications

A first map comprising local features and 3D locations of the local features is generated, the local features comprising visible features in a current image and a corresponding set of covisible features. A second map comprising prior features and 3D locations of the prior features may be determined, where each prior feature: was first imaged at a time prior to the first imaging of any of the local features, and lies within a threshold distance of at least one local feature. A first subset comprising previously imaged local features in the first map and a corresponding second subset of the prior features in the second map is determined by comparing the first and second maps, where each local feature in the first subset corresponds to a distinct prior feature in the second subset. A transformation mapping a subset of local features to a subset of prior features is determined.

Geometric matching in visual navigation systems

Embodiments disclosed pertain to the use of user equipment (UE) for the generation of a 3D exterior envelope of a structure based on captured images and a measurement set associated with each captured image. In some embodiments, a sequence of exterior images of a structure is captured and a corresponding measurement set comprising Inertial Measurement Unit (IMU) measurements, wireless measurements (including Global Navigation Satellite (GNSS) measurements) and/or other non-wireless sensor measurements may be obtained concurrently. A closed-loop trajectory of the UE in global coordinates may be determined and a 3D structural envelope of the structure may be obtained based on the closed loop trajectory and feature points in a subset of images selected from the sequence of exterior images of the structure.

Exterior hybrid photo mapping

An accelerometer in a mobile device is calibrated by taking multiple measurements of acceleration vectors when the mobile device is held stationary at different orientations with respect to a plane normal. A circle is calculated that fits respective tips of measured acceleration vectors in the accelerometer coordinate system. The radius of the circle and the lengths of the measured acceleration vectors are used to calculate a rotation angle for aligning the accelerometer coordinate system with the mobile device surface. A gyroscope in the mobile device is calibrated by taking multiple measurements of a rotation axis when the mobile device is rotated at different rates with respect to the rotation axis. A line is calculated that fits the measurements. The angle between the line and an axis of the gyroscope coordinate system is used to align the gyroscope coordinate system with the mobile device surface.

Method for aligning a mobile device surface with the coordinate system of a sensor

Example methods, apparatuses, or articles of manufacture are disclosed herein that may be utilized, in whole or in part, to facilitate or support one or more operations or techniques for reducing power consumption or error of a digital compass.

Reducing power consumption or error of digital compass

Embodiments disclosed obtain a plurality of measurement sets from a plurality of sensors in conjunction with the capture of a sequence of exterior and interior images of a structure while traversing locations in and around the structure. Each measurement set may be associated with at least one image. An external structural envelope of the structure is determined from exterior images of the structure and the corresponding outdoor trajectory of a UE. The position and orientation of the structure and the structural envelope is determined in absolute coordinates. Further, an indoor map of the structure in absolute coordinates may be obtained based on interior images of the structure, a structural envelope in absolute coordinates, and measurements associated with the indoor trajectory of the UE during traversal of the indoor area to capture the interior images.

Hybrid photo navigation and mapping

A homography between two captured images of a planar object is decomposed into at least one possible solution, and typically at least two ambiguous solutions. The ambiguity between the two solutions is removed, or a single solution validated, using measurements from orientation sensors. The measurements from orientation sensors may be used by comparing at least one of the yaw, pitch, and/or roll angles derived from a relative rotation matrix for the one or more solutions to a corresponding at least one of the yaw, pitch, and/or roll angles derived from the measurements from the orientation sensors.

Resolving homography decomposition ambiguity based on orientation sensors

Systems, apparatus and methods for estimating gravity and/or scale in a mobile device are presented. A difference between an image-based pose and an inertia-based pose is using to update the estimations of gravity and/or scale. The image-based pose is computed from two poses and is scaled with the estimation of scale prior to the difference. The inertia-based pose is computed from accelerometer measurements, which are adjusted by the estimation for gravity.

Adaptive scale and/or gravity estimation

A mobile device tracks a relative pose between a camera and a target using Vision aided Inertial Navigation System (VINS), that includes a contribution from inertial sensor measurements and a contribution from vision based measurements. When the mobile device detects movement of the target, the contribution from the inertial sensor measurements to track the relative pose between the camera and the target is reduced or eliminated. Movement of the target may be detected by comparing vision only measurements from captured images and inertia based measurements to determine if a discrepancy exists indicating that the target has moved. Additionally or alternatively, movement of the target may be detected using projections of feature vectors extracted from captured images.

Adaptive switching between vision aided INS and vision only pose

Growing community of inventors

San Diego, CA, United States of America

Christopher Brunner