Deepmap Inc.

Nvidia Corporation

Leica Geosystems Ag

Google Inc.

Leica Geosystems Hds, Inc.

Cyra Technologies, Inc.

Mark Damon Wheeler

Lin Yang

Richard William Bukowski

Chen Chen

Jonathan Apollo Kung

Eli Dylan Lorimer

Bret Edward Peterson

Vijay Raman

Manas Ranjan Jagadev

Di Zeng

Ronghua Zhang

Derik Schroeter

Jeffrey Minoru Adachi

Gregory William Coombe

Derek Thomas Miller

Gil Colgate

Liang Zou

Laura Michele Downs

Yu Zhang

Xiaqing Wu

Christopher Robin Thewalt

Wei Luo

Dimitrios Ioannou

Minghua Wu

Jeff Adachi

Zhiyun Li

David Kramer

Lenord Melvix Joseph Stephen Max

Ziqiang Huang

William Edward Wallace

Laslo Vespremi

Qian Gong

Zhengyu Zhang

Ramakrishnan Kazhiyur-Mannar

Barry Joel Wheeler

Dongzhen Piao

Barry Joel Schwarz

Daniel Chudak

Yuelin Chen

Greg Coombe

A system accesses a three-dimensional map of a geographic region and generates a two-dimensional projection of the road based on the three-dimensional map. The two-dimensional projection comprises a plurality of points along the road and each point is assigned a score measuring a navigability of the point. Based on the navigability score of each point and history of vehicle positions on the road, the system identifies a plurality of navigable points on the two-dimensional projection of the road. Based on the plurality of navigable points, the system determines a navigable surface corresponding to a physical area over which a vehicle may safely navigate and navigable surface boundaries surrounding that area. The navigable surface area and boundaries on the two-dimensional projection are converted into a three-dimensional representation, which the system uses to generate an updated three-dimensional map of the geographic region.

Navigable boundary generation for autonomous vehicles

A high-definition map system receives sensor data from vehicles travelling along routes and combines the data to generate a high definition map for use in driving vehicles, for example, for guiding autonomous vehicles. A pose graph is built from the collected data, each pose representing location and orientation of a vehicle. The pose graph is optimized to minimize constraints between poses. Points associated with surface are assigned a confidence measure determined using a measure of hardness/softness of the surface. A machine-learning-based result filter detects bad alignment results and prevents them from being entered in the subsequent global pose optimization. The alignment framework is parallelizable for execution using a parallel/distributed architecture. Alignment hot spots are detected for further verification and improvement. The system supports incremental updates, thereby allowing refinements of sub-graphs for incrementally improving the high-definition map for keeping it up to date.

Detection of misalignment hotspots for high definition maps for navigating autonomous vehicles

Embodiments relate to methods for efficiently encoding sensor data captured by an autonomous vehicle and building a high definition map using the encoded sensor data. The sensor data can be LiDAR data which is expressed as multiple image representations. Image representations that include important LiDAR data undergo a lossless compression while image representations that include LiDAR data that is more error-tolerant undergo a lossy compression. Therefore, the compressed sensor data can be transmitted to an online system for building a high definition map. When building a high definition map, entities, such as road signs and road lines, are constructed such that when encoded and compressed, the high definition map consumes less storage space. The positions of entities are expressed in relation to a reference centerline in the high definition map. Therefore, each position of an entity can be expressed in fewer numerical digits in comparison to conventional methods.

Encoding LiDAR scanned data for generating high definition maps for autonomous vehicles

According to an aspect of an embodiment, operations may comprise receiving a LIDAR scan of a scene from a LIDAR of a vehicle with the scene comprising a board, detecting the board in the LIDAR scan, fitting a plane through LIDAR coordinates corresponding to the detected board, projecting the plane from the LIDAR coordinates to a first set of camera coordinates, detecting the board in a camera image from a camera of the vehicle at a second set of camera coordinates, and calibrating the LIDAR of the vehicle and the camera of the vehicle by determining a transform between the first set of camera coordinates and the second set of camera coordinates.

LIDAR-to-camera transformation during sensor calibration for autonomous vehicles

A vehicle computing system validates location data received from a Global Navigation Satellite System receiver with other sensor data. In one embodiment, the system calculates velocities with the location data and the other sensor data. The system generates a probabilistic model for velocity with a velocity calculated with location data and variance associated with the location data. The system determines a confidence score by applying the probabilistic model to one or more of the velocities calculated with other sensor data. In another embodiment, the system implements a machine learning model that considers features extracted from the sensor data. The system generates a feature vector for the location data and determines a confidence score for the location data by applying the machine learning model to the feature vector. Based on the confidence score, the system can validate the location data. The validated location data is useful for navigation and map updates.

Augmentation of global navigation satellite system based data

According to an aspect of an embodiment, operations may comprise receiving an approximate geographic location of a vehicle, accessing a map of a region within which the approximate geographic location of the vehicle is located, identifying a first region on the map within a first threshold distance of the approximate geographic location of the vehicle, identifying a second region on the map associated with one or more roads on the map, determining a search space on the map within which the vehicle is likely to be present, the search space representing an intersection of the first region and the second region, and determining a more accurate geographic location of the vehicle by performing a search within the search space.

Using map-based constraints for determining vehicle state

An online system builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region. The autonomous vehicles detect map discrepancies based on differences in the surroundings observed using sensor data compared to the high definition map and send messages describing these map discrepancies to the online system. The online system updates existing occupancy maps to improve the accuracy of the occupancy maps (OMaps), and to thereby improve passenger and pedestrian safety. While vehicles are in motion, they can continuously collect data about their surroundings. When new data is available from the various vehicles within a fleet, this can be updated in a local representation of the occupancy map and can be passed to the online HD map system (e.g., in the cloud) for updating the master occupancy map shared by all of the vehicles.

Occupancy map updates based on sensor data collected by autonomous vehicles

High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.

High definition map and route storage management system for autonomous vehicles

A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

Calibrating sensors mounted on an autonomous vehicle

According to an aspect of an embodiment, operations may comprise capturing, at a vehicle as the vehicle travels, LIDAR scans and camera images. The operations may further comprise selecting, at the vehicle as the vehicle travels, a subset of the LIDAR scans and the camera images that are determined to be useful for calibration. The operations may further comprise computing, at the vehicle as the vehicle travels, LIDAR-to-camera transformations for the subset of the LIDAR scans and the camera images using an optimization algorithm. The operations may further comprise calibrating, at the vehicle as the vehicle travels, one or more sensors of the vehicle based on the LIDAR-to-camera transformations.

Online sensor calibration for autonomous vehicles

An autonomous vehicle system removes ephemeral points from lidar samples. The system receives a plurality of light detection and ranging (lidar) samples captured by a lidar sensor. Along with the lidar samples, the system receives an aligned pose and an unwinding transform for each of the lidar samples. The system determines one or more occupied voxel cells in a three-dimensional (3D) space using the lidar samples, their aligned poses, and their unwinding transforms. The system identifies occupied voxel cells representative of noise associated with motion of an object relative to the lidar sensor. The system filters the occupied voxel cells by removing the cells representative of noise. The system inputs the filtered occupied voxel cells in a 3D map comprising voxel cells, e.g., during the map generation and/or a map update.

Removal of ephemeral points from point cloud of a high-definition map for navigating autonomous vehicles

The autonomous vehicle generates an overlapped image by overlaying HD map data over sensor data and rendering the overlaid images. The visualization process is repeated as the vehicle drives along the route. The visualization may be displayed on a screen within the vehicle or at a remote device. The system performs reverse rendering of a scene based on map data from a selected point. For each line of sight originating at the selected point, the system identifies the farthest object in the map data. Accordingly, the system eliminates objects obstructing the view of the farthest objects in the HD map as viewed from the selected point. The system further allows filtering of objects using filtering criteria based on semantic labels. The system generates a view from the selected point such that 3D objects matching the filtering criteria are eliminated from the view.

Visualization of high definition map data

According to an aspect of an embodiment, operations may comprise determining a target position and orientation for a calibration board with respect to a camera of a vehicle, detecting a first position and orientation of the calibration board with respect to the camera of the vehicle, determining instructions for moving the calibration board from the first position and orientation to the target position and orientation, transmitting the instructions to a device, detecting a second position and orientation of the calibration board, determining whether the second position and orientation is within a threshold of matching the target position and orientation, and, in response to determining that the second position and orientation is within the threshold of matching the target position and orientation, capturing one or more calibration camera images using the camera and calibrating one or more sensors of the vehicle using the one or more calibration camera images.

Directing board repositioning during sensor calibration for autonomous vehicles

Growing community of inventors

Saratoga, CA, United States of America

Mark Damon Wheeler