Nvidia Corporation

Mengxi Wu

Di Zeng

Derik Schroeter

Operations may comprise obtaining a plurality of light detection and ranging (LIDAR) scans of a region. The operations may also comprise identifying a plurality of LIDAR poses that correspond to the plurality of LIDAR scans. In addition, the operations may comprise identifying, as a plurality of keyframes, a plurality of images of the region that are captured during capturing of the plurality of LIDAR scans. The operations may also comprise determining, based on the plurality of LIDAR poses, a plurality of camera poses that correspond to the keyframes. Further, the operations may comprise identifying a plurality of two-dimensional (2D) keypoints in the keyframes. The operations also may comprise generating one or more three-dimensional (3D) keypoints based on the plurality of 2D keypoints and the respective camera poses of the plurality of keyframes.

Image-based keypoint generation

According to an aspect of an embodiment, operations may comprise accessing a set of vehicle poses of one or more vehicles; for each of the set of vehicle poses, accessing a high definition (HD) map of a geographical region surrounding the vehicle pose, with the HD map comprising a three-dimensional (3D) representation of the geographical region, determining a measure of constrainedness for the vehicle pose, with the measure of constrainedness representing a confidence for performing localization for the vehicle pose based on 3D structures surrounding the vehicle pose, and storing the measure of constrainedness for the vehicle pose; and for each of the geographical regions surrounding each of the set of vehicle poses, determining a measure of constrainedness for the geographical region based on measures of constrainedness of vehicle poses within the geographical region, and storing the measure of constrainedness for the geographical region.

Using measure of constrainedness in high definition maps for localization of vehicles

Operations of the present disclosure include obtaining a first measure of velocity of a vehicle based on a plurality of locations determined for the vehicle. The operations also include obtaining, based on IMU measurements of an inertial measurement unit (IMU) of the vehicle, a second measure of velocity of the vehicle. In addition, the operations include performing calibration of the IMU based on the first measure of velocity and the second measure of velocity.

Calibration of inertial measurement units of vehicles using localization

According to an aspect of an embodiment, operations may comprise (a) accessing a portion of a high definition (HD) map comprising a point cloud of a region through which a vehicle is driving, (b) identifying a base LIDAR from a plurality of LIDARs mounted on the vehicle, (c) for each of the LIDARs: receiving a LIDAR scan comprising a point cloud of the region, and determining a pose for the LIDAR, (d) for each LIDAR other than the base LIDAR, determining a transform for the LIDAR with respect to the base LIDAR, (e) repeating (c) to generate a plurality of samples, (f) for each of the samples, repeating (d) to determine a plurality of transforms for each LIDAR with respect to the base LIDAR, and (g) calibrating each of the LIDARs other than the base LIDAR by determining an aggregate transform for the LIDAR.

Calibration of multiple lidars mounted on a vehicle using localization based on a high definition map

According to an aspect of an embodiment, operations may comprise receiving, from a LIDAR mounted on a vehicle, a first 3D point cloud comprising points of a region around the vehicle as observed by the LIDAR. The operations may also comprise accessing an HD map comprising a second 3D point cloud comprising points of the region around the vehicle. The operations may also comprise segmenting LIDAR ground points from LIDAR non-ground points in the first 3D point cloud. The operations may also comprise segmenting map ground points from map non-ground points in the second 3D point cloud. The operations may also comprise determining a pose of the vehicle by matching the LIDAR ground points to the map ground points and by matching the LIDAR non-ground points to the map non-ground points.

Segmenting ground points from non-ground points to assist with localization of autonomous vehicles

According to an aspect of an embodiment, operations may comprise obtaining a first point cloud that includes a first point. The operations also comprises obtaining a second point cloud that is a copy of the first point cloud and that includes a second point that is a copy of the first point. The operations also comprises moving the second point cloud with respect to the first point cloud according to a first vector. The operations also comprises identifying a closest point of the first point cloud that is closest to the second point of the second point cloud. The operations also comprises determining a second vector between the closest point and the second point. The operations also comprises determining a measure of usefulness of the first point based on the first vector and the second vector. The operations also comprises indicating the measure of usefulness of the first point.

Growing community of inventors

Mountain View, CA, United States of America

Mengxi Wu