Gm Global Technolgoy Operations, Inc.

Scale Ai, Inc.

Elliot Branson

Adam Samuel Cadien

Eric Lujan

Andrea Allais

Micah Christopher Chambers

William Gongshu Xie

Benjamin Earle Weinstein-Raun

Stuart Glaser

Peter Ming Gao

Chiao-Lun Cheng

Juan Fasola

Kyle Vogt

Shiva Ghose

Gabriel Warshauer-Baker

Gautier Minster

Leigh Marie Braswell

Mark Liu

Siddhartho Bhattacharya

Zhichang Yan

Zhizhong Yan

Shariq Shahab Hashme

Haggai Megged Nuchi

Suchir Balaji

Simon Alexander Hewat

Nathan Hayflick

Nathan Herman

Daniel Havír

Daniel Havíř

Jeffrey Alan Anders

An autonomous vehicle is described herein. The autonomous vehicle comprises a first sensor and a second sensor having limited fields of view, an articulation system, and a computing system. The computing system determines a first region and a second region external to the autonomous vehicle based on a sensor prioritization scheme comprising a ranking of regions surrounding the autonomous vehicle. The computing system then causes the articulation system to orient the first sensor towards the first region and the second region towards the second region. Responsive to receiving a sensor signal from the first sensor indicating that an object has entered a field of view of the first sensor, the computing system determines a third region having a higher ranking than the second region within the sensor prioritization scheme. The computing system then causes the articulation system to orient the second sensor towards the third region.

Controlling articulating sensors of an autonomous vehicle

An autonomous vehicle is described herein. The autonomous vehicle includes a lidar sensor system. The autonomous vehicle additionally includes a computing system that executes a lidar segmentation system, wherein the lidar segmentation system is configured to identify objects that are in proximity to the autonomous vehicle based upon output of the lidar sensor system. The computing system further includes a deep neural network (DNN), where the lidar segmentation system identifies the objects in proximity to the autonomous vehicle based upon output of the DNN.

Autonomous vehicle controlled based upon a lidar data segmentation system

One embodiment provides techniques for automatically pre-labeling point cloud data with cuboid annotations. Point cloud data is processed using ML models to detect, associate, and localize objects therein, in order to generate cuboid tracks that each include a series of cuboid annotations associated with an object. An object detection model that detects objects and performs coarse localization is trained using a loss function that separately evaluates the distances between corners of predicted cuboids and corners of ground truth cuboids for position, size, and yaw. A refinement model that performs more accurate localization takes as input 2D projections of regions surrounding cuboid tracks predicted by the object detection model and the cuboid tracks, and outputs refined cuboid tracks. The refined cuboid tracks are filtered to a set of keyframes, with in-between frames being interpolated. The cuboid tracks can then be presented to a user for viewing and editing.

Pre-labeling data with cuboid annotations

One embodiment of the present invention sets forth a technique for performing a labeling task. The technique includes generating a multi-scale representation of an image as input to a machine learning model. The technique also includes performing one or more operations that apply the machine learning model to the multi-scale representation of the image to produce a semantic segmentation comprising predictions of labels for regions of pixels in the image. The technique further includes outputting, in a user interface, the semantic segmentation for use in assisting a user in specifying the labels for the pixels in the image.

Prelabeling for semantic segmentation tasks

Various technologies described herein pertain to controlling an autonomous vehicle to provide indicators that signal a driving intent of the autonomous vehicle. The autonomous vehicle includes a plurality of sensor systems that generate a plurality of sensor signals, a notification system, and a computing system. The computing system determines that the autonomous vehicle is to execute a maneuver that will cause the autonomous vehicle to traverse a portion of a driving environment of the autonomous vehicle. The computing system predicts that a person in the driving environment is to traverse the portion of the driving environment based upon the plurality of sensor signals. The computing system then controls the notification system to output a first indicator indicating that the autonomous vehicle plans to yield to the person or a second indicator indicating that the autonomous vehicle plans to execute the maneuver prior to the person traversing the portion of the driving environment.

Autonomous vehicle intent signaling

Autonomous vehicle controlled based upon a LIDAR data segmentation system

An autonomous vehicle is described herein. The autonomous vehicle includes several different types of sensor systems, such as image, lidar, and radar. The autonomous vehicle additionally includes a computing system that executes several different object classifier modules, wherein the object classifier modules are configured to identify types of objects that are in proximity to the autonomous vehicle based upon outputs of the sensor systems. The computing system additionally executes a Bayesian object classifier system that is configured to receive outputs of the object classifier modules and assign labels to objects captured in sensor signals based upon the outputs of the object classifier modules.

Autonomous vehicle that is controlled based upon output of a Bayesian object classifier system

A processor-implemented method in a vehicle for detecting objects from radar data includes: retrieving radar measurements taken at different periodic time increments; organizing the radar measurements into appropriate time windows; building a sequence cluster of radar measurements wherein the sequence cluster comprises a sliding window the latest time windows of radar measurements; removing noise from the sequence cluster of radar measurements by removing a cluster of radar measurements from the sequence cluster of radar measurements that is contradictory to a road topology map for an area in which the first object is estimated to be situated; and outputting the sequence cluster of radar measurements after removal of contradictory radar measurements as a new cluster of radar measurements.

Systems and methods for using road understanding to constrain radar tracks

An autonomous vehicle is described herein. The autonomous vehicle includes a lidar sensor system. The autonomous vehicle additionally includes a computing system that executes a lidar segmentation system, wherein the lidar segmentation system is configured to identify objects that are in proximity to the autonomous vehicle based upon output of the lidar sensor system. The computing system further includes a deep neural network (DNN), where the lidar segmentation system identifies the objects in proximity to the autonomous vehicle based upon output of the DNN. The computing system is further configured to align a heightmap to lidar data output by the lidar sensor system based upon output of the DNN. The lidar segmentation system can identify objects in proximity to the autonomous vehicle based upon the aligned heightmap.

Autonomous vehicle controlled based upon a lidar data segmentation system and an aligned heightmap

Systems and method are provided for controlling a vehicle. In one embodiment, a localization method includes receiving sensor data relating to an environment of a vehicle, the sensor data including a plurality of sensor returns associated with objects in the environment, each of the sensor returns having a plurality of corresponding attributes, and constructing a first plurality of sensor data groups, each including a self-consistent subset of the plurality of sensor returns based on their corresponding attributes. The method further includes defining, for each of the first plurality of sensor data groups, a first set of features, wherein each feature is based on at least one of the corresponding attributes and each has an associated feature location, and determining, with a processor, a feature correlation between the first set of features and a second, previously determined set of features.

Systems and methods for radar localization in autonomous vehicles

Methods and systems are provided for tracking an object. The system includes a data receiving module receiving two dimensional imaging data including an object and height map data correlating ground height and location. A two dimensions to three dimensions transformation module determines a location of the object in three dimensional space based on the two dimensional imaging data and the height map data. A tracking module tracks the object using the location of the object.

Object tracking

A method for image-based vehicle localization includes measuring, at a vehicle, a position and a heading of the vehicle; capturing, at an image capture system of the vehicle, road surface image data of a road surface; processing the road surface image data to correct for distortion in the road surface image data due to pitch and roll of the vehicle; performing feature detection on the processed road surface image data to detect lane markers on the road surface; generating a local map based on the detected lane markers; wherein generating the local map comprises identifying a lane demarcated by the detected lane markers; and controlling the vehicle according to the local map.

Growing community of inventors

San Francisco, CA, United States of America

Elliot Branson