Aurora Operations, Inc.

Uatc, LLC

Xuanyuan Tu

Raquel Urtasun

Mengye Ren

Sergio Casas

Sivabalan Manivasagam

Jingkang Wang

Shun Da Suo

Davi Eugenio Nascimento Frossard

Ming Liang

Bin Yang

Abbas Sadat

Tsun-Hsuan Wang

Ava Alison Pun

Nicholas Baskar Vadivelu

Systems and methods for streaming sensor packets in real-time are provided. An example method includes obtaining a sensor data packet representing a first portion of a three-hundred and sixty degree view of a surrounding environment of a robotic platform. The method includes generating, using machine-learned model(s), a local feature map based at least in part on the sensor data packet. The local feature map is indicative of local feature(s) associated with the first portion of the three-hundred and sixty degree view. The method includes updating, based at least in part on the local feature map, a spatial map to include the local feature(s). The spatial map includes previously extracted local features associated with a previous sensor data packet representing a different portion of the three-hundred and sixty degree view than the first portion. The method includes determining an object within the surrounding environment based on the updated spatial map.

Systems and methods for sensor data packet processing and spatial memory updating for robotic platforms

Systems and methods for vehicle-to-vehicle communications are provided. An adverse system can obtain sensor data representative of an environment proximate to a targeted system. The adverse system can generate an intermediate representation of the environment and a representation deviation for the intermediate representation. The representation deviation can be designed to disrupt a machine-learned model associated with the target system. The adverse system can communicate the intermediate representation modified by the representation deviation to the target system. The target system can train the machine-learned model associated with the target system to detect the modified intermediate representation. Detected modified intermediate representations can be discarded before disrupting the machine-learned model.

Systems and methods for training machine-learned models with deviating intermediate representations

Techniques for generating testing data for an autonomous vehicle (AV) are described herein. A system can obtain sensor data descriptive of a traffic scenario. The traffic scenario can include a subject vehicle and actors in an environment. Additionally, the system can generate a perturbed trajectory for a first actor in the environment based on perturbation values. Moreover, the system can generate simulated sensor data. The simulated sensor data can include data descriptive of the perturbed trajectory for the first actor in the environment. Furthermore, the system can provide the simulated sensor data as input to an AV control system. The AV control system can be configured to process the simulated sensor data to generate an updated trajectory for the subject vehicle in the environment. Subsequently, the system can evaluate an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value.

Generating autonomous vehicle testing data through perturbations and adversarial loss functions

Systems and methods for improved vehicle-to-vehicle communications are provided. A system can obtain sensor data depicting its surrounding environment and input the sensor data (or processed sensor data) to a machine-learned model to perceive its surrounding environment based on its location within the environment. The machine-learned model can generate an intermediate environmental representation that encodes features within the surrounding environment. The system can receive a number of different intermediate environmental representations and corresponding locations from various other systems, aggregate the representations based on the corresponding locations, and perceive its surrounding environment based on the aggregated representations. The system can determine relative poses between the each of the systems and an absolute pose for each system based on the representations. Each representation can be aggregated based on the relative or absolute poses of each system and weighted according to an estimated accuracy of the location corresponding to the representation.

Systems and methods for mitigating vehicle pose error across an aggregated feature map

Systems and methods for training object detection models using adversarial examples are provided. A method includes obtaining a training scene and identifying a target object within the training scene. The method includes obtaining an adversarial object and generating a modified training scene based on the adversarial object, the target object, and the training scene. The modified training scene includes the training scene modified to include the adversarial object placed on the target object. The modified training scene is input to a machine-learned model configured to detect the training object. A detection score is determined based on whether the training object is detected, and the machine-learned model and the parameters of the adversarial object are trained based on the detection output. The machine-learned model is trained to maximize the detection output. The parameters of the adversarial object are trained to minimize the detection output.

Systems and methods for training object detection models using adversarial examples

Growing community of inventors

Milton, Canada

Xuanyuan Tu