Motional Ad LLC

Alex Hunter Lang

Oscar Olof Beijbom

Sourabh Vora

Bassam Helou

Paul Schmitt

Holger Caesar

Varun Bankiti

Jugal Buddhadev

Whye Kit Fong

Qiang Xu

Eric Wolff

Elena Corina Grigore

Cheng Jiang

Yitian Wu

Provided are methods for customized tags for annotating sensor data, which can include receiving sensor data captured during a plurality of sensor data capture sessions, processing the sensor data using a plurality of machine learning models to identify a plurality of capture session collections represented in the sensor data, filtering the sensor data based at least partly on a user-specified category of the plurality of categories of capture session to identify a capture session collection, of the plurality of capture session collections, representing sensor data of one or more sensor data capture sessions that conforms to the user-specified category, and transmitting the sensor data of one or more sensor data capture sessions that conforms to the user-specified category to an end user computing device. Systems and computer program products are also provided.

Adding tags to sensor data via a plurality of models and querying the sensor data

In an embodiment, a method includes: receiving ambient sound; determining if the ambient sound includes a siren; in accordance with determining that the ambient sound includes a siren, determining a first location associated with the siren; receiving a camera image; determining if the camera image includes a flashing light; in accordance with determining that the camera image includes a flashing light, determining a second location associated with the flashing light; 3D data; determining if the 3D data includes an object; in accordance with determining that the 3D data includes an object, determining a third location associated with the object; determining a presence of an emergency vehicle based on the siren, detected flashing light and detected object; determining an estimated location of the emergency vehicle based on the first, second and third locations; and initiating an action related to the vehicle based on the determined presence and location.

Emergency vehicle detection system and method

Provided are methods for motion prediction in an autonomous vehicle using fused synthetic and camera images. The method can include obtaining data pairs, each of which reflects data corresponding to a synthetic image representing a birds-eye-view of an area around a vehicle and identifying an object, and data corresponding to a camera image depicting the object. A machine learning model can be trained based on the data pairs to result in a trained model that predicts motion of the object within the data pair based on the synthetic image and camera image in the data pair. Systems and computer program products are also provided.

Motion prediction in an autonomous vehicle using fused synthetic and camera images

Among other things, we describe techniques for detecting objects in the environment surrounding a vehicle. A computer system is configured to receive a set of measurements from a sensor of a vehicle. The set of measurements includes a plurality of data points that represent a plurality of objects in a 3D space surrounding the vehicle. The system divides the 3D space into a plurality of pillars. The system then assigns each data point of the plurality of data points to a pillar in the plurality of pillars. The system generates a pseudo-image based on the plurality of pillars. The pseudo-image includes, for each pillar of the plurality of pillars, a corresponding feature representation of data points assigned to the pillar. The system detects the plurality of objects based on an analysis of the pseudo-image. The system then operates the vehicle based upon the detecting of the objects.

Deep learning for object detection using pillars

Provided are methods for end-to-end perception system training using fused images, which can include fusing different types of images to form a fused image, extracting features from the fused image, calculating a loss, and modifying at least one network parameter of an image semantic network based on the loss. Systems and computer program products are also provided.

End-to-end system training using fused images

Techniques are provided for improving a perception processing pipeline for object detection that fuses image segmentation data (e.g., segmentation scores) with LiDAR points. The disclosed techniques are implemented using an architecture that accepts point clouds and images as input and estimates oriented 3D bounding boxes for all relevant object classes. In an embodiment, a method comprises: matching temporally, using one or more processors of a vehicle, points in a three-dimensional (3D) point cloud with an image; generating, using an image-based neural network, semantic data for the image; decorating, using the one or more processors, the points in the 3D point cloud with the semantic data; and estimating, using a 3D object detector with the decorated points as input, oriented 3D bounding boxes for the one or more objects.

Sequential fusion for 3D object detection

In an embodiment, a method comprises: receiving ambient sound; determining if the ambient sound includes a siren; in accordance with determining that the ambient sound includes a siren, determining a first location associated with the siren; receiving a camera image; determining if the camera image includes a flashing light; in accordance with determining that the camera image includes a flashing light, determining a second location associated with the flashing light; 3D data; determining if the 3D data includes an object; in accordance with determining that the 3D data includes an object, determining a third location associated with the object; determining a presence of an emergency vehicle based on the siren, detected flashing light and detected object; determining an estimated location of the emergency vehicle based on the first, second and third locations; and initiating an action related to the vehicle based on the determined presence and location.

Growing community of inventors

Culver City, CA, United States of America

Alex Hunter Lang