Zoox, Inc.

Gerrit Bagschik

Andrew Scott Crego

James William Vaisey Philbin

Ashutosh Gajanan Rege

Marc Wimmershoff

Andreas Christian Reschka

Andraz Kavalar

Antoine Ghislain Deux

Rodin Lyasoff

Kai Zhenyu Wang

Siavosh Rezvan Behbahani

Gowtham Garimella

Sai Anurag Modalavalasa

Michael Haggblade

Mahsa Ghafarianzadeh

Kshitij Agarwal

Aditya Pramod Khadilkar

Thomas Bernard Gacka

Muhammad Farooq Rama

Anne-Claire Elisabeth Marie Le Henaff

Julia A Wyatt

Pengzhi Tian

Techniques are discussed herein for generating and evaluating autonomous vehicle control systems. In examples, a driving scenario is determined and log data, e.g., generated from user-supported vehicles travelling in an environment, may be used to determine instances in which vehicles experience the driving scenario. Based on the log data, an exposure metric, a severity metric, and/or a controllability metric may be determined. A safety goal can be determined from the metric(s) and vehicle control parameters are determined to comport with the safety goal. The vehicle control parameters may be used to control an autonomous vehicle in the environment.

Safe vehicle operation using log data

Techniques for top-down scene discrimination are discussed. A system receives scene data associated with an environment proximate a vehicle. The scene data is input to a convolutional neural network (CNN) discriminator trained using a generator and a classification of the output of the CNN discriminator. The CNN discriminator generates an indication of whether the scene data is a generated scene or a captured scene. If the scene data is data generated scene, the system generates a caution notification indicating that a current environmental situation is different from any previous situations. Additionally, the caution notification is communicated to at least one of a vehicle system or a remote vehicle monitoring system.

Top-down scene discrimination

Techniques for top-down scene generation are discussed. A generator component may receive multi-dimensional input data associated with an environment. The generator component may generate, based at least in part on the multi-dimensional input data, a generated top-down scene. A discriminator component receives the generated top-down scene and a real top-down scene. The discriminator component generates binary classification data indicating whether an individual scene in the scene data is classified as generated or classified as real. The binary classification data is provided as a loss to the generator component and the discriminator component.

Top-down scene generation

Techniques are discussed herein for determining truncated simulation regions within a parameter space of simulation scenarios, such as driving scenarios used to analyze and evaluate the responses of autonomous vehicle controllers. Using non-sampling-based parameter selection techniques, parameterized scenarios may be executed as simulations to determine the truncated simulation region. Sampling-based parameter selection techniques may be used to determine additional parameterized scenarios, which may be compared to the truncated simulation region. Parameterized scenarios within the truncated simulation region may be executed as simulations and scenarios outside of the truncated simulation region may be excluded, and the aggregated results may be analyzed to determine scenario/vehicle performance metrics across the scenario parameter space.

Scenario simulation execution within a truncated parameter space

Techniques for determining an error model based on vehicle data and ground truth data are discussed herein. To determine whether a complex system (which may be not capable of being inspected) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data. To provide safe operation of such a system, an error model can be determined that can provide a probability associated with perception data and a vehicle can determine a trajectory based on the probability of an error associated with the perception data.

Perception error models

Techniques associated with generating simulation scenarios for simulating a vehicle controller are discussed herein. Log data may include sensor data captured by sensors of a vehicle. The log data may represent objects in an environment. Objects may be associated with a region of a discretized representation of the environment relative to the vehicle. Specific states of objects (relative position in a region type, velocity, classification, size, etc.) may represent an instance of an occupation. Log data can be aggregated based on similar region type and/or object state. A statistical model over object states can be determined for each region type and can later be sampled to determine simulation parameters. A simulation scenario can be generated based on the simulation parameters, and a vehicle controller can be evaluated based on the simulation scenario.

System for generating generalized simulation scenarios

Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.

Safety analysis framework

Techniques for determining an error model associated with a system/subsystem of vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, errors can be introduced into operating regimes (scenarios) to validate the safe operation of such a system. By comparing captured and/or generated vehicle data with ground truth data, an error of the system can be statistically quantified and modeled. The statistical model can be used to introduce errors to the scenario to perturb the scenario to test, for example, a vehicle controller. Based on a simulation of the vehicle controlled in the perturbed scenario, a safety metric associated with the vehicle controller can be determined, as well as causes for any failures.

Error modeling framework

Simulating realistic movement of an object, such as a vehicle or pedestrian, that accounts for unusual behavior may comprise generating an agent behavior model based at least in part on output of a perception component of an autonomous vehicle and determining a difference between the output and log data that includes indications of an actual maneuver of location of an object. Simulating movement of an object may comprise determining predicted motion of the object using the perception component and modifying the predicted motion based at least in part on the agent behavior model.

Growing community of inventors

Foster City, CA, United States of America

Gerrit Bagschik