Gm Cruise Holdings LLC

Pranay Agrawal

Matthew Cox

Changkai Zhou

Yong Jae Lee

Ashish Shrivastava

Chiyu Jiang

Robert Liang

Systems and techniques are provided for processing sensor data using a machine learning model. An example method can include receiving, by a machine learning model configured to perform object detection, sensor data corresponding to one or more sensors on an autonomous vehicle; determining, based on the sensor data, a first object prediction by a first detection head of the machine learning model, wherein the first detection head is configured to generate the first object prediction based on a first input from a first sensor from the one or more sensors; and determining, based on the sensor data, a second object prediction by a second detection head of the machine learning model, wherein the second detection head is configured to generate the second object prediction based on a fusion of the first input from the first sensor and a second input from a second sensor from the one or more sensors.

Multi-head machine learning model for processing multi-sensor data

The disclosed technology provides solutions for applying a flare detection model to an image for detecting road flares in an environment by an autonomous vehicle (AV). A method comprises accessing a flare detection model that is trained to recognize a bounded outer region and a bounded inner region; and detect a flare object within the bounded inner region and refrain from detecting an object in the bounded outer region; applying the flare detection model to an image of the environment to recognize an outer region corresponding to a glow of a flare and an inner region that corresponds to the flare in the environment; and detecting the flare in the environment within the inner region based on the inner region being recognized by the flare detection model and contained within the outer region that is also recognized by the flare detection model. Systems and machine-readable media are also provided.

Method for visual detection and position estimation of road flares

Methods and systems are provided for detecting objects by utilizing uncertainties. In some aspects, a process can include steps for receiving, by an autonomous vehicle system, a frame of a scene with a detected object; estimating, by the autonomous vehicle system, an overall probability of the detected object in the frame; estimating, by the autonomous vehicle system, covariances for each state of at least one bounding box; and balancing, by the autonomous vehicle system, confidence values of the at least one bounding box based on the overall probability of the detected object and the covariances of each state of the at least one bounding box.

Estimating object uncertainty using a pre-non-maximum suppression ensemble

Various technologies described herein pertain to detecting data in a lidar point cloud representative of vapor and controlling an autonomous vehicle based on such detection. The lidar point cloud is outputted by a lidar sensor system of the autonomous vehicle. The techniques set forth herein utilize dual return lidar data outputted by the lidar sensor system. The dual return lidar data includes two lidar returns received responsive to a light beam emitted (e.g., at a particular azimuthal angle) by the lidar sensor system into a driving environment of the autonomous vehicle. The data in the lidar point cloud detected as being caused by vapor can be removed from downstream processing; accordingly, the autonomous vehicle can be controlled such that the autonomous vehicle need not stop for or maneuver around vapor in the driving environment.

Vapor detection in lidar point cloud

The present disclosure provides a perception system for a vehicle. The perception system includes a perception filter for determining a region of interest ('ROI') for the vehicle based on an intent of the vehicle and a current state of the vehicle; and a perception module for perceiving an environment of the vehicle based on the ROI; wherein the vehicle is caused to take appropriate action based on the perceived environment, the current state of the vehicle, and the intent of the vehicle.

Intent-based dynamic change of region of interest of vehicle perception system

An AV is described herein. The AV includes a lidar sensor system. The AV additionally includes a computing system that executes a road surface analysis component to determine, based upon lidar sensor data, whether a road surface feature is present on or in a roadway in a travel path of the AV. The AV can be configured to initiate a mitigation maneuver responsive to determining that the road surface feature is present. Performing the mitigation maneuver causes the AV to avoid the road surface feature or decelerate prior to reaching the road surface feature, thereby improving the apparent quality or comfort of the ride to a passenger of the AV.

Lidar based detection of road surface features

Various technologies described herein pertain to detecting sensor data to be annotated for autonomous vehicle perception algorithm training. A label identifying a type of an object is assigned to an object at a particular location in an environment based on sensor data generated by a sensor system of an autonomous vehicle for a given time. The label is assigned based on a confidence score assigned to the type of the object by a computer-implemented perception algorithm. The computer-implemented perception algorithm assigns the confidence score to the type of the object based on the sensor data corresponding to the particular location in the environment for the given time. An output of a heuristic is generated based on the label and/or the confidence score, and the output of the heuristic is used to control whether to cause the sensor data for the given time to be annotated.

Automatic detection of data for annotation for autonomous vehicle perception

The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter determines compute resource priority instructions based on an intent of the vehicle and a current state of the vehicle; and a compute module for receiving the compute resource priority instructions from the perception filter and allocating compute resources among the imaging devices in accordance with the compute resource priority instructions.

Intent-based dynamic change of compute resources of vehicle perception system

The present disclosure provides a perception system for a vehicle. The perception system includes a number of imaging devices associated with the vehicle and a perception filter for determining a region of interest ('ROI') for the vehicle based on an intent of the vehicle and a current state of the vehicle. The ROI is used to select images of an environment of the vehicle produced by the imaging devices and the perception filter receives and processes the images produced by the imaging device. The perception system further includes a perception module for receiving the processed images from the perception filter and perceiving the environment of the vehicle based at least in part on the received images.

The present disclosure provides perception system for a vehicle that includes a plurality of imaging devices for producing images of an environment of the vehicle; a perception filter for receiving the images produced by the imaging devices, wherein the perception filter crops and filters the received images based on an intent of the vehicle and a current state of the vehicle; and a perception module for receiving at least one of the cropped and filtered images from the perception filter and perceiving the environment of the vehicle based on the received at least one of the cropped and filtered images.

Growing community of inventors

San Francisco, CA, United States of America

Pranay Agrawal