Motional Ad LLC

Uber Technologies, Inc.

Aptiv Technologies Limited

Yimu Wang

Junqing Wei

Wenda Xu

Jarrod M Snider

Guchan Ozbilgin

Yifan Yang

Ajay Charan

Wanzhi Zhang

Carlos Vallespi-Gonzalez

Paul Schmitt

Ankit Laddha

J Andrew Bagnell

Yih-Jye Hsu

Varun Ramakrishna

Ning Xu

Provided are methods for semantic annotation of sensor data using unreliable map annotation inputs, which can include training a machine learning model to accept inputs including images representing sensor data for a geographic area and unreliable semantic annotations for the geographic area. The machine learning model can be trained against validated semantic annotations for the geographic area, such that subsequent to training, additional images representing sensor data and additional unreliable semantic annotations can be passed through the neural network to provide predicted semantic annotations for the additional images. Systems and computer program products are also provided.

Vehicle location using combined inputs of redundant localization pipelines

Among other things, techniques are described for identifying sensor data from a sensor of a first vehicle that includes information related to a pose of at least two other vehicles on a road. The technique further includes determining a geometry of a portion of the road based at least in part on the information about the pose of the at least two other vehicles. The technique further includes comparing the geometry of the portion of the road with map data to identify a match between the portion of the road and a portion of the map data. The technique further includes determining a pose of the first vehicle relative to the map data based at least in part on the match.

Localization based on surrounding vehicles

A sensor data fusion system for a vehicle with multiple sensors includes a first-sensor, a second-sensor, and a controller-circuit. The first-sensor is configured to output a first-frame of data and a subsequent-frame of data indicative of objects present in a first-field-of-view. The first-frame is characterized by a first-time-stamp, the subsequent-frame of data characterized by a subsequent-time-stamp different from the first-time-stamp. The second-sensor is configured to output a second-frame of data indicative of objects present in a second-field-of-view that overlaps the first-field-of-view. The second-frame is characterized by a second-time-stamp temporally located between the first-time-stamp and the subsequent-time-stamp. The controller-circuit is configured to synthesize an interpolated-frame from the first-frame and the subsequent-frame. The interpolated-frame is characterized by an interpolated-time-stamp that corresponds to the second-time-stamp. The controller-circuit fuses the interpolated-frame with the second-frame to provide a fused-frame of data characterized by the interpolated-time-stamp, and operates the host-vehicle in accordance with the fused-frame.

Data fusion system for a vehicle equipped with unsynchronized perception sensors

Among other things, techniques are described for identifying, in a light detection and ranging (LiDAR) scan line, a first LiDAR data point and a plurality of LiDAR data points within a vicinity of the first LiDAR data point. The techniques may further include identifying, based on a comparison of the first LiDAR data point to at least one LiDAR data point of the plurality of LiDAR return points, a coefficient of the first LiDAR data point, wherein the coefficient is related to image smoothness. The techniques may further include identifying, based on a comparison of the coefficient to a threshold, whether to include the first LiDAR data point in an updated LiDAR scan line, and then identifying, based on the updated LiDAR scan line, a location of the autonomous vehicle.

Light detection and ranging (LiDaR) scan smoothing

Provided are methods for deep learning-based camera calibration, which can include receiving first and second images captured by a camera, processing the first image using a first neural network to determine a depth of the first image, processing the first image and the second image using a second neural network to determine a transformation between a pose of the camera for the first image and a pose of the camera for the second image, generating a projection image based on the depth of the first image, the transformation of the pose of the camera, and intrinsic parameters of the camera, comparing the second image and the projection image to determine a reprojection error, and adjusting at least one of the intrinsic parameters of the camera based on the reprojection error. Systems and computer program products are also provided.

Deep learning-based camera calibration

A vehicle perception sensor adjustment system includes a perception-sensor, a digital-map, and controller-circuit. The perception-sensor is configured to detect an object proximate to a host-vehicle. The perception-sensor is characterized as having a field-of-view that is adjustable. The digital-map indicates a contour of a roadway traveled by the host-vehicle. The controller-circuit in communication with the perception-sensor and the digital-map. The controller-circuit determines the field-of-view of the perception-sensor in accordance with the contour of the roadway indicated by the digital-map, and outputs a control-signal to the perception-sensor that adjusts the field-of-view of the perception-sensor.

Vehicle sensor configuration based on map data

Systems and methods for detecting and classifying objects that are proximate to an autonomous vehicle can include receiving, by one or more computing devices, LIDAR data from one or more LIDAR sensors configured to transmit ranging signals relative to an autonomous vehicle, generating, by the one or more computing devices, a data matrix comprising a plurality of data channels based at least in part on the LIDAR data, and inputting the data matrix to a machine-learned model. A class prediction for each of one or more different portions of the data matrix and/or a properties estimation associated with each class prediction generated for the data matrix can be received as an output of the machine-learned model. One or more object segments can be generated based at least in part on the class predictions and properties estimations. The one or more object segments can be provided to an object classification and tracking application.

Growing community of inventors

Pittsburgh, PA, United States of America

Yimu Wang