Nvidia Corporation

Deepmap Inc.

Derik Schroeter

Di Zeng

Mengxi Wu

Mark Damon Wheeler

Chen Chen

Gregory William Coombe

Jeffrey Minoru Adachi

Greg Coombe

Liang Zou

According to one or more embodiments, operations may comprise obtaining a first point cloud. The operations also comprise performing segmentation of the first point cloud, the segmentation generating one or more clusters of points of the point cloud. The operations also comprise determining, for each respective cluster of the plurality of clusters, a respective geometric feature of a corresponding object that corresponds to the respective cluster. The operations also comprise obtaining a second point cloud. The operations also comprise assigning a plurality of weights that comprises assigning a respective weight to each respective cluster based on the respective geometric feature that corresponds to the respective cluster. The operations also comprise obtaining a second point cloud and aligning the first point cloud with the second point cloud based on the plurality of weights.

Determining weights of points of a point cloud based on geometric features

Operations may comprise obtaining a plurality of light detection and ranging (LIDAR) scans of a region. The operations may also comprise identifying a plurality of LIDAR poses that correspond to the plurality of LIDAR scans. In addition, the operations may comprise identifying, as a plurality of keyframes, a plurality of images of the region that are captured during capturing of the plurality of LIDAR scans. The operations may also comprise determining, based on the plurality of LIDAR poses, a plurality of camera poses that correspond to the keyframes. Further, the operations may comprise identifying a plurality of two-dimensional (2D) keypoints in the keyframes. The operations also may comprise generating one or more three-dimensional (3D) keypoints based on the plurality of 2D keypoints and the respective camera poses of the plurality of keyframes.

Image-based keypoint generation

According to an aspect of an embodiment, operations may comprise receiving an approximate geographic location of a vehicle, accessing a map of a region within which the approximate geographic location of the vehicle is located, identifying a first region on the map within a first threshold distance of the approximate geographic location of the vehicle, identifying a second region on the map associated with one or more roads on the map, determining a search space on the map within which the vehicle is likely to be present, the search space representing an intersection of the first region and the second region, and determining a more accurate geographic location of the vehicle by performing a search within the search space.

Using map-based constraints for determining vehicle state

According to an aspect of an embodiment, operations may comprise accessing a set of vehicle poses of one or more vehicles; for each of the set of vehicle poses, accessing a high definition (HD) map of a geographical region surrounding the vehicle pose, with the HD map comprising a three-dimensional (3D) representation of the geographical region, determining a measure of constrainedness for the vehicle pose, with the measure of constrainedness representing a confidence for performing localization for the vehicle pose based on 3D structures surrounding the vehicle pose, and storing the measure of constrainedness for the vehicle pose; and for each of the geographical regions surrounding each of the set of vehicle poses, determining a measure of constrainedness for the geographical region based on measures of constrainedness of vehicle poses within the geographical region, and storing the measure of constrainedness for the geographical region.

Using measure of constrainedness in high definition maps for localization of vehicles

An autonomous vehicle system removes ephemeral points from lidar samples. The system receives a plurality of light detection and ranging (lidar) samples captured by a lidar sensor. Along with the lidar samples, the system receives an aligned pose and an unwinding transform for each of the lidar samples. The system determines one or more occupied voxel cells in a three-dimensional (3D) space using the lidar samples, their aligned poses, and their unwinding transforms. The system identifies occupied voxel cells representative of noise associated with motion of an object relative to the lidar sensor. The system filters the occupied voxel cells by removing the cells representative of noise. The system inputs the filtered occupied voxel cells in a 3D map comprising voxel cells, e.g., during the map generation and/or a map update.

Removal of ephemeral points from point cloud of a high-definition map for navigating autonomous vehicles

According to an aspect of an embodiment, operations may comprise receiving, from a LIDAR mounted on a vehicle, a first 3D point cloud comprising points of a region around the vehicle as observed by the LIDAR. The operations may also comprise accessing an HD map comprising a second 3D point cloud comprising points of the region around the vehicle. The operations may also comprise segmenting LIDAR ground points from LIDAR non-ground points in the first 3D point cloud. The operations may also comprise segmenting map ground points from map non-ground points in the second 3D point cloud. The operations may also comprise determining a pose of the vehicle by matching the LIDAR ground points to the map ground points and by matching the LIDAR non-ground points to the map non-ground points.

Segmenting ground points from non-ground points to assist with localization of autonomous vehicles

According to an aspect of an embodiment, operations may comprise receiving a search query for points near a query-point, accessing a compressed octree representation of a point cloud comprising 3D points of a region, and traversing the compressed octree representation to identify regions that overlap a search space by, marking a current node as overlapping the search space responsive to determining that the current node is a leaf node, identifying a child node of the current node and performing a nearest neighbor search in the child node responsive to determining that a region represented by the current node overlaps the search space, and identifying a sibling node of the current node and performing the nearest neighbor search in the sibling node responsive to determining that a region represented by the current node does not overlap the search space.

Nearest neighbor search using compressed octrees representing high definition maps for autonomous vehicles

A system align point clouds obtained by sensors of a vehicle using kinematic iterative closest point with integrated motions estimates. The system receives lidar scans from a lidar mounted on the vehicle. The system derives point clouds from the lidar scan data. The system iteratively determines velocity parameters that minimize an aggregate measure of distance between corresponding points of the plurality of pairs of points. The system iteratively improves the velocity parameters. The system uses the velocity parameters for various purposes including for building high definition maps used for navigating the vehicle.

Iterative closest point process based on lidar with integrated motion estimation for high definition maps

According to an aspect of an embodiment, operations may comprise for each of the set of geographic X-positions, accessing an HD map of a geographical region surrounding the geographic X-position, determining a convergence range for the geographic X-position, and storing the convergence range for the geographic X-position in the HD map. The operations may also comprise accessing the HD map, predicting a next geographic X-position of a target vehicle, predicting a covariance of the predicted next geographic X-position, accessing the convergence range for the geographic X-position in the HD map closest to the predicted next geographic X-position, estimating a current geographic X-position of the target vehicle by performing a localization algorithm, and determining a confidence value for the estimated current geographic X-position of the target vehicle based on the predicted next geographic X-position, the predicted covariance, and the accessed convergence range.

Determining localization confidence of vehicles based on convergence ranges

Operations may comprise obtaining a first point cloud from a map representing a region. The operations may also include obtaining a second point cloud from one or more sensors of a vehicle traveling through the region. In addition, the operations may include identifying one or more subsets of clusters of second points of the second point cloud. The operations may also include determining correspondences between first points of the first point cloud and cluster points of the one or more subsets of clusters of the second point cloud. Moreover, the operations may include identifying at least a cluster of the one or more subsets of clusters, the identified cluster having, with respect to first points of the first point cloud, a correspondence percentage that is less than a threshold value. The operations may also include adjusting the second point cloud based on the identified cluster.

Identifying dynamic objects in a point cloud

A high-definition map system receives sensor data from vehicles travelling along routes and combines the data to generate a high definition map for use in driving vehicles, for example, for guiding autonomous vehicles. A pose graph is built from the collected data, each pose representing location and orientation of a vehicle. The pose graph is optimized to minimize constraints between poses. Points associated with surface are assigned a confidence measure determined using a measure of hardness/softness of the surface. A machine-learning-based result filter detects bad alignment results and prevents them from being entered in the subsequent global pose optimization. The alignment framework is parallelizable for execution using a parallel/distributed architecture. Alignment hot spots are detected for further verification and improvement. The system supports incremental updates, thereby allowing refinements of subgraphs for incrementally improving the high-definition map for keeping it up to date.

Growing community of inventors

Fremont, CA, United States of America

Derik Schroeter