Zoox, Inc.

Gary Linscott

Ashutosh Gajanan Rege

Jesse Sol Levinson

Marin Kobilarov

Ravi Verma Gogna

Paul Orecchio

Dan Xie

Amanda Lee Kelly Lockwood

Timothy Caldwell

Rachad Youssef Gamara

Jacob Lee Askeland

Ryan Martin Cahoon

Timothy David Kentley-Klay

Gabriel Thurston Sibley

Peter Scott Schleede

Joseph Funke

Ethan Miller Pronovost

Robert Edward Somers

Joona Markus Petteri Kiiski

Adrien Treuille

Timothy David Kentley

Gowtham Garimella

Jefferson Bradfield Packer

Andreas Pasternak

Ke Sun

Xiangyu Xie

Aaron Huang

Seungji Lee

Yan Chang

A machine-learned model that uses sensor and/or perception data to directly determine controls for operating an autonomous vehicle may be trained by identifying a preferred trajectory between a human-driven and vehicle-controlled trajectory, and using a first loss determined between the vehicle-controlled trajectory and the path the autonomous vehicle ultimately ended up taking in a scenario and a second loss determined between the vehicle-controlled trajectory and the human-driven trajectory to refine the machine-learned model. The machine-learned model may additionally or alternatively be refined by a learned reward model constructed by replacing one or more output heads of the machine-learned model with a regression head that is trained using performance metrics determined for the vehicle-controlled trajectory.

Refinement training for machine-learned vehicle control model

Techniques for generating a tree structure based on multiple machine-learned trajectories are described herein. A planning component ('ML system') within a vehicle may receive and encode various types of sensor and/or vehicle data. The ML system can provide the encoded data as input to multiple machine-learning models (“ML models”), each of which may be trained to output a unique candidate trajectory for the vehicle follow. In some examples, each ML model may be trained to output a unique type of learned trajectory that causes the vehicle to perform a certain type of action. Using the learned candidate trajectories, the ML system may generate a tree structure that includes some or all of the candidate trajectories. The vehicle may determine a control trajectory based on the generation and traversal of the tree structure using a tree search algorithm, and may follow the control trajectory within the environment.

Vehicle trajectory tree structure including learned trajectories

A machine-learned architecture for estimating the cost determined by a cost function for a prediction node of a tree search for exploring potential paths for controlling a vehicle may include two portions: a set up portion that includes models trained to process static data and a second portion that processes dynamic object data. The respective portions of the architecture may comprise various models that determine intermediate outputs that may be projected into a space associated with estimated cost. That estimated cost may identify an estimate of an output of the cost function for paths that are based on a particular prediction node of the tree search.

Machine-learned cost estimation in tree search trajectory generation for vehicle control

A machine-learned architecture for generating a single trajectory or multiple trajectories for controlling a vehicle may comprise an embedding model that generates an embedding of a world state indicating environment, object, and/or other states, one or more machine-learned layers that determine a predicted world state embedding using the world state embedding, and concatenating, as combined data, that predicted world state embedding to a steering angle distribution and a velocity distribution. The combined data is provided as input to a machine-learned model (that may be a single machine-learned model or may comprise two separate machine-learned models) that determines a next steering angle distribution and a next velocity distribution. These distributions may be used as part of generating one or more trajectories for controlling the vehicle. The architecture may be iteratively used to create a series of distributions that are used to create one or more trajectories.

Vehicle control using auto-regressive control distibution generation based on world state embeddings

Techniques for improving operational decisions of an autonomous vehicle are discussed herein. In some cases, a system may generate reference graphs associated with a route of the autonomous vehicle. Such reference graphs can comprise precomputed feature vectors based on grid regions and/or lane segments. The feature vectors are usable to determine scene context data associated with static objects to reduce computational expenses and compute time.

System for generating scene context data using a reference graph

An imitation learning-based machine-learned (ML) model to augment or replace the prediction and/or planner components of an autonomous vehicle may be trained using a two stage and multi-discipline approach. A first stage of training may include training the ML component to output a predicted action associated with a target vehicle and modifying the ML component to reduce a difference between the predicted action and the observed action taken by the target vehicle. A second stage may use reinforcement learning to further tun the ML component. The resultant model may be used on its own, with enough training data, or to rank or weight candidate trajectories generated by a planning component of the vehicle. The ML component may provide embeddings of environment features to first transformer(s) that output to a long short-term memory that outputs to second transformer(s) to determine the predicted action.

Machine-learned component hybrid training and assistance of vehicle trajectory generation

Trajectory generation and/or execution architecture is described. In an example, a first signal can be determined at a first frequency, wherein the first signal comprises information associated with causing the system to move to a location. Further, a second signal can be determined at a second frequency different from the first frequency and based at least in part on the first signal. A system can be controlled to move to the location, based at least in part on the second signal.

Trajectory generation and execution architecture

Automating reinforcement learning for autonomous vehicles may include assigning a probability with a scenario and varying that probability based at least in part on changes in performance by the autonomous vehicle associated with that scenario. The amount of time and computational bandwidth required to train a machine-learned component of an autonomous vehicle and the accuracy of the machine-learned component may be improved by determining a reward for performance of the autonomous vehicle in a scenario based at least in part on an severity metric. The impact severity metric may be determined based at least in part on a velocity, angle, and/or interaction area associated with the impact.

Automated reinforcement learning scenario variation and impact penalties

A domain specific language for use in constructing simulations within real environments is described. In an example, a computing device associated with a vehicle can receive, from one or more sensors associated with the vehicle, sensor data associated with an environment within which the vehicle is positioned. In an example, the vehicle can be an autonomous vehicle. The computing device associated with the vehicle can receive simulated data associated with one or more primitives that are to be instantiated as a scenario in the environment. The computing device can merge the sensor data and the simulated data to generate aggregated data and determine a trajectory along which the vehicle is to drive based at least in part on the aggregated data. The computing device can determine instructions for executing the trajectory and can assess the performance of the vehicle based on how the vehicle responds to the scenario.

Onboard use of scenario description language

A method for autonomously operating a driverless vehicle along a path between a first geographic location and a destination may include receiving communication signals from the driverless vehicle. The communication signals may include sensor data from the driverless vehicle and data indicating occurrence of an event associated with the path. The communication signals may also include data indicating that a confidence level associated with the path is less than a threshold confidence level due to the event. The method may also include determining, via a teleoperations system, a level of guidance to provide the driverless vehicle based on data associated with the communication signals, and transmitting teleoperations signals to the driverless vehicle. The teleoperations signals may include guidance to operate the driverless vehicle according to the determined level of guidance, so that a vehicle controller maneuvers the driverless vehicle to avoid, travel around, or pass through the event.

Interactions between vehicle and teleoperations system

A driverless vehicle may include a processor, a sensor, a network interface, and a memory having stored thereon processor-executable instructions. The driverless vehicle may be configured to obtain a stream of sensor signals including sensor data related to operation of the driverless vehicle from the sensor and/or the network interface. The driverless vehicle may be configured to determine a confidence level associated with operation of the driverless vehicle from the sensor data, and store the confidence level and at least a portion of the sensor data. The driverless vehicle may also be configured to transmit via the network interface a request for teleoperator assistance, and the request may include the portion of the sensor data and the confidence level.

Teleoperator situational awareness

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. More specifically, systems, devices, and methods are configured to initiate modification of trajectories to influence navigation of autonomous vehicles. In particular, a method may include receiving a teleoperation message via a communication link from an autonomous vehicle, detecting data from the teleoperation message specifying an event associated with the autonomous vehicle, identifying one or more courses of action to perform responsive to detecting the data specifying the event, and generating visualization data to present information associated with the event to a display of a teleoperator computing device.

Teleoperation system and method for trajectory modification of autonomous vehicles

A domain specific language, or Scenario Description Language (SDL), can be used for quickly enumerating scenarios in a simulation for testing and validating interaction of an object (e.g., an autonomous vehicle) within an environment. Scenarios in a simulation are defined using one or more primitives. Primitives are used to define objects to be instantiated (such as body size, position, orientation, velocities, etc.) and/or actions to be performed by the objects in the simulation (such as wait for a period of time, goal positions, follow a particular object, etc.). The SDL enables simple creation of multiple scenarios by combining primitives combinatorially and in some examples, limiting which scenarios are created to those that correspond to combinations that provide meaningful information. Additionally, the SDL allows for instantiation to be agnostic of map features so that a particular scenario can be instantiated automatically over all possible positions within a map.

Scenario description language

A teleoperator device may be configured to obtain a request for teleoperator assistance from a driverless vehicle and obtain teleoperator data in response to the request. The teleoperator device may also be configured to record at least some of the teleoperator input and/or guidance transmitted to the driverless vehicle based on the teleoperator input. Upon receiving a subsequent request, the teleoperator device may be configured to reproduce at least part of the former teleoperator input and/or to provide an option to activate guidance associated with the teleoperator input. The teleoperator device may also be configured to train a model and/or use a model to determine from vehicle data an option for presentation via a teleoperator interface and/or a presentation configuration of the teleoperator interface.

Growing community of inventors

Cupertino, CA, United States of America

Gary Linscott