Zoox, Inc.

Prateek Chandresh Shah

Janek Hudecek

Peter Scott Schleede

Jeremy Schwartz

Eric Chu

Wentao Zhong

Rick Zhang

Leonardo Poubel Orenstein

Xiaosi Zeng

Deepan Subrahmanian Palguna

Robert Jonathan Crane

John Connelly Kegelman

Tracking perception and/or planning component decisions may comprise generating a data structure in association with an output determined by a component. This data structure, along with one or more data structures generated in association with other outputs generated by the same or different components of the vehicle, may be used to determine a trace that identifies component(s) that determined outputs that affected a particular component's generation of an output. This trace and/or factors identified by the data structure may be used to detect a problem with a component; identify a component or output therefrom that may be causing an error; determine a portion of a component that has been activated and a frequency associated therewith; and/or visualize operation of the vehicle associated with activation of particular components, among additional or alternate uses discussed herein.

Vehicle operation and/or simulation decision registry

Techniques for determining classifications associated with vehicle behavior by a machine learned model are discussed herein. A computing device can combine a variety of classifier labels as input to train a machine learned model to determine classifications that represent a vehicle behavior as either good behavior or bad behavior. The machine learned model may receive weak classifier labels that classify an aspect of a vehicle behavior as 'good,' “bad,” or “unknown” (relative to the classification) to generate training data for training the machine learning model. Data associated with a vehicle or simulated vehicle may be input to the trained machine learned model to classify good vehicle behavior or bad vehicle behavior even when the model has not been exposed to the precise scenarios represented in the training data.

Training vehicle behavior labels

Tracking component decisions may comprise generating a data structure in association with an output determined by a component. This data structure, along with one or more data structures generated in association with other outputs generated by the same or different components of the vehicle, may be used to determine a trace that identifies component(s) that determined outputs that affected a particular component's generation of an output. The data structures and/or traces may be used to determine whether a component is the source of an error, a portion of the component that is the source of the error, unintended impacts to unmodified portions of components, among additional or alternate uses discussed herein.

Vehicle operation and/or simulation based on decision registry

Techniques for aggregating costs associated with one or more heat maps to control a vehicle in an environment are discussed herein. A vehicle computing device can implement a model to determine heat maps and respective cost information for different features of the environment based on sensor data. The vehicle computing device can output a planned trajectory for the vehicle based on combining the heat maps. The techniques can also include determining a rationalization or root cause detailing reasons why the planned trajectory was determined.

Optimization based planning system

A vehicle computing system may implement techniques to improve collision prediction and avoidance between a vehicle and objects in an environment. A vehicle computing system of the vehicle generates a relevance polygon associated with a planned path of the vehicle based at least in part on a speed associated with the vehicle traveling through the environment. The vehicle computing system identifies objects in the environment and determines whether the objects are located within a boundary of the relevance polygon. Based on a determination that an object is within the boundary, the vehicle computing system determines that the object is relevant to the vehicle and includes data associated therewith in vehicle control planning considerations.

Velocity-based relevance filter

Techniques for generating simulations to evaluate an update to a controller. The controller may be configured to control one or more functionalities of an autonomous and/or a semi-autonomous vehicle. A simulation computing system may receive a request to evaluate a first controller. The simulation computing system may generate a simulation based on data associated with a previous operation of the vehicle in an environment, the previous operation being controlled by a second controller (e.g., standard for evaluation, control version, etc.). The simulation computing device may cause the first controller to control a simulated vehicle in the simulation and may determine whether to validate the update to the controller based on a difference between first metrics associated with a control of the simulated vehicle by the first controller and second metrics associated with a control of the vehicle by the second controller.

Vehicle controller validation

This application relates to techniques for determining whether to engage an autonomous controller of a vehicle based on previously recorded data. A computing system may receive, from a vehicle computing system, data representative of a vehicle being operated in an environment, such as by an autonomous controller. The computing system may generate a simulation associated with the vehicle operation and configured to test an updated autonomous controller. The computing system may determine one or more first time periods associated with the vehicle operations that satisfy one or more conditions associated with engaging an autonomous controller and one or more second time periods associated with the vehicle operations that fail to satisfy the one or more conditions. The computing system may enable an engagement of the autonomous controller during the one or more first time periods and disable the engagement during the one or more second time periods.

Autonomous control engagement

A vehicle may include an active ride comfort tuning system that reactively and/or proactively alters a parameter of a system of the autonomous vehicle to mitigate or avoid interruptions to ride smoothness. For example, the comfort tuning system may alter a parameter of a drive system, suspension, and/or a trajectory cost function. The comfort tuning system may alter the parameter based at least in part on detecting and/or receiving a comfort indication, determined based on sensor data, user input, or the like.

Vehicle ride dynamics active comfort tuning system

A vehicle may include an active ride comfort tuning system that reactively and/or proactively alters a parameter of a system of the autonomous vehicle to mitigate or avoid interruptions to ride smoothness. An indication that a ride smoothness interruption occurred at a vehicle may be generated at the vehicle and/or at a remote computing device. The remote computing device may determine an altered parameter for controlling how the vehicle operates and a ruleset for when the vehicle should implement the altered parameter in place of a nominal parameter.

Vehicle fleet remote ride comfort tuning management system

This application relates to techniques for dynamically determining whether to engage an autonomous controller of a vehicle. A computing system may receive a request to engage the autonomous controller (e.g., autonomous mode) of the vehicle. In some examples, the request may be received from a simulation computing system configured to test an updated autonomous controller in a simulation. Based on a determination that conditions associated with engaging autonomy are satisfied, the computing system engages the autonomous controller. Based on a determination that conditions associated with engaging autonomy are not satisfied, the computing system disables the engagement of the autonomous controller such that the vehicle is controlled according to an initial operational mode (e.g., manual mode, semi-autonomous mode, previous version of the autonomous controller, etc.).

Dynamic autonomous control engagement

Techniques for controlling a vehicle based on a collision avoidance algorithm are discussed herein. The vehicle receives sensor data and can determine that the sensor data represents an object in an environment through which the vehicle is travelling. A computing device associated with the vehicle determines a collision probability between the vehicle and the object at predicted locations of the vehicle and object at a first time. Updated locations of the vehicle and object can be determined, and a second collision probability can be determined. The vehicle is controlled based at least in part on the collision probabilities.

Collision avoidance planning system

Determining whether another entity is coordinating with an autonomous vehicle and/or to what extent the other entity's behavior is based on the autonomous vehicle may comprise determining a collaboration score and/or negotiation score based at least in part on sensor data. The collaboration score may indicate an extent to which the entity is collaborating with the autonomous vehicle to navigate (e.g., a likelihood that the entity is increasingly yielding the right of way to the autonomous vehicle based on the autonomous vehicle's actions). A negotiation score may indicate an extent to which behavior exhibited by the entity is based on actions of the autonomous vehicle (e.g., how well the autonomous vehicle and the entity are communicating with their actions).

Growing community of inventors

San Francisco, CA, United States of America

Prateek Chandresh Shah