Aurora Operations, Inc.

Google Inc.

Arun Venkatraman

James Andrew Bagnell

Sanjiban Choudhury

Venkatraman Narayanan

Haoyang Fan

Michael Lee Phillips

Paul Vernaza

Michael Bode

Micol Marchetti-Bowick

J Andrew Bagnell

Sumit Kumar

Yuhang Ma

Xinyan Yan

Pengju Jin

Juan Pablo Mendoza

Carl Wellington

Sanjiban Choudry

The present disclosure provides an example method that includes: (a) obtaining context data descriptive of an environment surrounding an autonomous vehicle, the context data based on map data and perception data; (b) generating, by a proposer and based on the context data: (i) a plurality of candidate trajectories, and (ii) a plurality of actor forecasts for a plurality of actors in the environment; (c) generating, by a ranker and based on the context data, the plurality of candidate trajectories, and the plurality of actor forecasts, a ranking of the plurality of candidate trajectories; and (d) controlling a motion of the autonomous vehicle based on a candidate trajectory selected based on the ranking of the plurality of candidate trajectories, wherein the proposer comprises a first machine-learned model and the ranker comprises a second machine-learned model, and wherein the first machine-learned model and the second machine-learned model use a common backbone architecture.

Autonomous vehicle motion planning

Techniques are disclosed for training one or more cost functions of an autonomous vehicle ('AV') control system based on difference between data generated using the AV control system and manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, multiple AV control systems can be evaluated by comparing deviations for each AV control system, where the deviations are determined using the same set of manual driving data.

Training machine learning model for controlling autonomous vehicle

Evaluating the performance of an autonomous vehicle includes determining a plurality of simulation scenarios, determining a set of features correlated to a performance metric of interest for the autonomous vehicle, executing a simulation for each simulation scenario in the plurality of simulation scenarios, determining, by a machine learning model, a weight for each simulation scenario in the set of simulation scenarios subject to a constraint that a simulated expected value of each feature in the plurality of simulation scenarios falls within a threshold range of an observed expected value of each feature in an operation design domain of interest of the autonomous vehicle, and estimating an expected value of the performance metric of interest of the autonomous vehicle based on the determined weight and the execution of the simulation for each simulation scenario in the plurality of simulation scenarios.

Estimating autonomous vehicle performance metrics in real world from simulation scenarios

Implementations process, using machine learning (ML) layer(s) of ML model(s), actor(s) from a past episode of locomotion of a vehicle and stream(s) in an environment of the vehicle during the past episode to forecast associated trajectories, for the vehicle and for each of the actor(s), with respect to a respective associated stream of the stream(s). Further, implementations process, using a stream connection function, the associated trajectories to forecast a plurality of associated trajectories, for the vehicle and each of the actor(s), with respect to each of the stream(s). Moreover, implementations iterate between using the ML layer(s) and the stream connection function to update the associated trajectories for the vehicle and each of the actor(s). Implementations subsequently use the ML layer(s) in controlling an AV.

Systems and methods related to controlling autonomous vehicle(s)

Techniques are disclosed for evaluating manual driving data using an AV control system based on differences between data generated using the AV control system and the manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, an additional AV control system can be trained using the evaluated manual driving data.

Evaluating driving data using autonomous vehicle control system

An example method includes obtaining log data descriptive of an exemplar action of an exemplar vehicle in an environment, the exemplar action occurring in an initial state of the environment; determining, using the operational system, a planned action for a simulated vehicle in the initial state of the environment; simulating an SUT state of the environment resulting from the simulated vehicle executing the planned action in the initial state of the environment and an actor performing an actor action subsequent to the planned action and an exemplar state of the environment resulting from the simulated vehicle executing the exemplar action in the initial state of the environment and the actor performing the actor action subsequent to the exemplar action; determining a test score based on the SUT state and a reference score based on the exemplar state; evaluating the operational system based on the test score and the reference score.

Systems and methods for autonomous vehicle validation

Systems and methods for training a machine learning ('ML') model for use in controlling an autonomous vehicle (“AV”) are described herein. Implementations can obtain an initial state instance from driving of a vehicle, obtain ground truth label(s) for subsequent state instance(s) that each indicate a corresponding action of the vehicle for a corresponding time instance, perform, for a given time interval, a simulated episode, of locomotion of a simulated AV, generate, for each of a plurality of time instances of the given time interval, subsequent simulated state instance(s) that differ from the subsequent state instance(s), determine, using the ML model, and for each of the time instances, a predicted simulated action of the simulated AV based on the subsequent simulated operation instance(s), generate loss(es) based on the predicted simulated actions and the ground truth labels, and update the ML model based on the loss(es).

Training machine learning model(s), in simulation, for use in controlling autonomous vehicle(s)

Techniques are disclosed for evaluating an autonomous vehicle ('AV') control system by determining deviations between data generated using the AV control system and manual driving data. In many implementations, manual driving data captures action(s) of a vehicle controlled by a manual driver. Additionally or alternatively, multiple AV control systems can be evaluated by comparing deviations for each AV control system, where the deviations are determined using the same set of manual driving data.

Growing community of inventors

Mountain View, CA, United States of America

Arun Venkatraman