Zoox, Inc.

Andreas Christian Reschka

Andrew Scott Crego

Collin MacGregor

Sai Anurag Modalavalasa

Ali Ghasemzadehkhoshgroudi

Siavosh Rezvan Behbahani

Lingqiao Qin

Rodin Lyasoff

Antoine Ghislain Deux

James William Vaisey Philbin

Ashutosh Gajanan Rege

Marc Wimmershoff

Gerrit Bagschik

Sy Kelly Olson

Joseph Funke

Chuang Wang

Ralph Michael Kling

Jefferson Bradfield Packer

Robert Edward Somers

Andrew Lewis King

Guillermo Duenas Arana

Yu Liu

Thomas Bernard Gacka

Gonzalo Javier Rey

Collin Macgregor

Techniques and methods for performing collision monitoring using error models. For instance, a vehicle may generate sensor data using one or more sensors. The vehicle may then analyze the sensor data using systems in order to determine parameters associated with the vehicle and parameters associated with another object. Additionally, the vehicle may process the parameters associated with the vehicle using error models associated with the systems in order to determine a distribution of estimated locations associated with the vehicle. The vehicle may also process the parameters associated with the object using the error models in order to determine a distribution of estimated locations associated with the object. Using the distributions of estimated locations, the vehicle may determine the probability of collision between the vehicle and the object.

Collision monitoring using statistic models

The present disclosure is directed to performing one or more validity checks on potential trajectories for a device, such as an autonomous vehicle, to navigate. In some examples, a potential trajectory may be validated based on whether it is consistent with a current trajectory the vehicle is navigating such that the potential and current trajectories are not too different, whether the vehicle can feasibly or kinematically navigate to the potential trajectory from a current state, whether the potential trajectory was punctual or received within a time period of a prior trajectory, and/or whether the potential trajectory passes a staleness check, such that it was created within a certain time period. In some examples, determining whether a potential trajectory is feasibly may include updating a set of feasibility limits based on one or more operational characteristics of statuses of subsystems of the vehicle.

Modifying limits on vehicle dynamics for trajectories

Testing autonomous vehicle control systems in the real world can be difficult, because creating and re-creating physical scenarios for repeated testing may be impractical. In some implementations, detailed map data and data acquired through driving in a region can be used to identify similar segments of a drivable surface. Simulation scenarios used to test one of the similar segments may be used to test other of the similar segments. The driving data may also be used to generate and/or validate the simulation scenarios, e.g., by re-creating scenarios encountered while driving in a first segment in a simulation scenario for use in the second segment and comparing simulated driving behavior with the driving data.

Simulating autonomous driving using map data and driving data

Performance anomalies in autonomous vehicle can be difficult to identify, and the impact of such anomalies on systems within the autonomous vehicle may be difficult to understand. In examples, systems of the autonomous vehicle are modeled as nodes in a probabilistic graphical network. Probabilities of data generated at each of the nodes is determined. The probabilities are used to determine capabilities associated with higher level functions of the autonomous vehicle.

Methods and systems to assess vehicle capabilities

Techniques for determining an error model based on vehicle data and ground truth data are discussed herein. To determine whether a complex system (which may be not capable of being inspected) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data. To provide safe operation of such a system, an error model can be determined that can provide a probability associated with perception data and a vehicle can determine a trajectory based on the probability of an error associated with the perception data.

Perception error models

Techniques and methods for performing collision monitoring using system data. For instance, a vehicle may generate sensor data using one or more sensors. The vehicle may then analyze the sensor data using systems in order to determine parameters associated with the vehicle and parameters associated with another object. Additionally, the vehicle may determine uncertainties associated with the parameters and then process the parameters using the uncertainties. Based at least in part on the processing, the vehicle may determine a distribution of estimated locations associated with the vehicle and a distribution of estimated locations associated with the object. Using the distributions of estimated locations, the vehicle may determine the probability of collision between the vehicle and the object.

Collision monitoring using system data

Techniques for predicting safety metrics associated with near-miss conditions for a vehicle, such as an autonomous vehicle, are discussed herein. For instance, a training system identifies an object in an environment and determines a trajectory for the object. The training system may receive a trajectory for a vehicle and associate the trajectory for the object and the trajectory for the vehicle with an event involving the object and the vehicle. In examples, the training system determines a parameter associated with motion of the vehicle as indicated by the trajectory of the vehicle relative to the trajectory of the object, and the event. Then, the training system may determine a safety metric associated with the event that indicates whether the vehicle came within a threshold of a collision with the object during a time period associated with the event.

Safety metric prediction

Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.

Safety analysis framework

Techniques and methods for training and/or using a machine learned model that identifies unsafe events. For instance, computing device(s) may receive input data, such as vehicle data generated by one or more vehicles and/or simulation data representing a simulated environment. The computing device(s) may then analyze features represented by the input data using one or more criteria in order to identify potential unsafe events represented by the input data. Additionally, the computing device(s) may receive ground truth data classifying the identified events as unsafe events or safe events. The computing device(s) may then train the machine learned model using at least the input data representing the unsafe events and the classifications. Next, when the computing device(s) and/or vehicles receive input data, the computing device(s) and/or vehicles may use the machine learned model to determine if the input data represents unsafe events.

Event detection based on vehicle data

A vehicle may include a primary system for generating data to control the vehicle and a secondary system that validates the data and/or other data to avoid collisions. For example, the primary system may localize the vehicle, detect an object around the vehicle, predict an object trajectory, and generate a trajectory for the vehicle. The secondary system may localize the vehicle, detect an object around the vehicle, predict an object trajectory, and determine a likelihood of a collision of the vehicle with the object. A simulation system may generate simulation scenarios that test aspects of the primary system and the secondary system. Simulation scenarios may include simulated vehicle control data that causes the primary system to generate a driving trajectory and simulated object data that causes the secondary system to determine a collision.

Systems and methods for testing collision avoidance systems

System and method for trajectory validation

Techniques for determining an error model associated with a system/subsystem of vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, errors can be introduced into operating regimes (scenarios) to validate the safe operation of such a system. By comparing captured and/or generated vehicle data with ground truth data, an error of the system can be statistically quantified and modeled. The statistical model can be used to introduce errors to the scenario to perturb the scenario to test, for example, a vehicle controller. Based on a simulation of the vehicle controlled in the perturbed scenario, a safety metric associated with the vehicle controller can be determined, as well as causes for any failures.

Error modeling framework

Autonomous vehicles use accurate and detailed maps for navigation. Expanding functionality of an autonomous vehicle to a new, e.g., unmapped, region can include determining drivable surface segments of the new region and comparing the segments to segments or classes of segments from an already-mapped region. Segments of the new region that are similar to segments from the mapped region can be identified as potentially navigable. An autonomous vehicle can travel through the new region via those segments indicated as navigable. In addition, during travel through the new region, data may be collected using sensors on the autonomous vehicle to map additional portions of the region and/or confirm a driving ability in the new region. Functionality of an autonomous vehicle may be limited based on how similar the segments are to one another.

Extension of autonomous driving functionality to new regions

A secondary system operates on a vehicle to avoid a collision when a problem occurs with a primary system. For example, the secondary system may operate independently from the primary system to take over control of the vehicle from the primary system when the secondary system detects a potential collision, when an error occurs with the primary system, and so on. In examples, the primary system may implement first techniques, such as Artificial Intelligence (AI) techniques, to understand an environment around the vehicle and/or instruct the vehicle to move within the environment. In examples, the secondary system may implement second techniques that are based on positioning, velocity, acceleration, etc. of the vehicle and/or objects around the vehicle.

Growing community of inventors

Foster City, CA, United States of America

Andreas Christian Reschka