Zoox, Inc.

Magna Electronics Inc.

Marc Wimmershoff

James William Vaisey Philbin

Sarah Tariq

Ashutosh Gajanan Rege

Jefferson Bradfield Packer

Andrew Scott Crego

Abishek Krishna Akella

Andreas Christian Reschka

Gerrit Bagschik

Antoine Ghislain Deux

Rodin Lyasoff

Timothy David Kentley-Klay

Subasingha Shaminda Subasingha

Marin Kobilarov

Joseph Funke

Janek Hudecek

Robert Edward Somers

Andrew Lewis King

Sai Anurag Modalavalasa

Marcus Hammond

Shayan Darayan

William Joseph Chundrlik, Jr

Ravi Verma Gogna

William Anthony Silva

Kristofer Sven Smeds

Techniques for detecting and responding to an emergency vehicle are discussed. A vehicle computing system may determine that an emergency vehicle based on sensor data, such as audio and visual data. In some examples, the vehicle computing system may determine aggregate actions of objects (e.g., other vehicles yielding) proximate the vehicle based on the sensor data. In such examples, a determination that the emergency vehicle is operating may be based on the actions of the objects. The vehicle computing system may, in turn, identify a location to move out of a path of the emergency vehicle (e.g., yield) and may control the vehicle to the location. The vehicle computing system may determine that the emergency vehicle is no longer relevant to the vehicle and may control the vehicle along a route to a destination. Determining to yield and/or returning to a mission may be confirmed by a remote operator.

Emergency vehicle detection and response

Command determination for controlling a vehicle, such as an autonomous vehicle, is described. In an example, individual requests for controlling the vehicle relative to each of multiple objects or conditions in an environment are received (substantially simultaneously) and based on the request type and/or additional information associated with a request, command controllers can determine control commands (e.g., different accelerations, steering angles, steering rates, and the like) associated with each of the one or more requests. The command controllers may have different controller gains (which may be based on functions of distance, distance ratios, time to estimated collisions, etc.) for determining the controls and a control command may be determined based on the all such determined controls.

Vehicle control

A vehicle can include a primary computing device and a secondary computing device. The primary computing device can receive a trajectory and can generate control data to control the vehicle based on a computed state. Further, the primary computing device can send the internal data to the secondary computing device configured to control the vehicle in the event of a failure of the primary computing device. The secondary computing device can receive the internal data as first internal data and determine a capability associated with the primary computing device. Using the first internal data, the secondary computing device can determine second internal data and, based on the capability (e.g., in event of a failure of the primary computing device), can control the vehicle to follow a trajectory using the second internal data. Transferring state between an active to a standby computing device can ensure algorithmic synchronization and safe operation.

Multi-controller synchronization

Energy consumption for a vehicle may be reduced based at least in part on an environment characteristic associated with the environment through which the vehicle travels or an operation characteristic associated with operation of the vehicle, thereby increasing an operational time of the vehicle. In some situations, reducing energy consumption may be associated with operation of one or more of a sensor (e.g., turning the sensor off, reducing a frequency or resolution of the sensor, etc.) and/or one or more processors associated with the vehicle (e.g., turning a processor off, reducing a rate of computation, etc.) based at least in part on one or more of the environment characteristic signal or the operation characteristic signal.

Energy consumption control systems and methods for vehicles

A vehicular vision system includes a camera disposed at an in-cabin side of a windshield of a vehicle. Responsive at least in part to processing of captured image data, the system determines a camera-derived path of travel of the vehicle along a road. Responsive at least in part to a geographic location of the vehicle, the system determines a geographic-derived path of travel of the vehicle along the road. Control of the vehicle along the road is based on diminished reliance on the determined geographic-derived path of travel when a geographic location reliability level of the determined geographic-derived path of travel is below a threshold geographic location reliability level. Control of the vehicle along the road is based on diminished reliance on the determined camera-derived path of travel of the vehicle when a camera reliability level of the determined camera-derived path of travel is below a threshold camera reliability level.

Vehicular system for controlling vehicle

A vehicle may include a primary system and a secondary system to validate operation of the primary system and to control the vehicle to avoid collisions. For example, the secondary system may receive multiple trajectories from the primary system, such as a primary trajectory and a secondary, contingent, trajectory associated with a deceleration or other maneuver. The secondary system may determine if a trajectory is associated with a potential collision, if the trajectory is consistent with a current or previous pose, if the trajectory is compatible with a capability of the vehicle, etc. The secondary system may select the primary trajectory if valid, the secondary trajectory if the primary trajectory is invalid, or another trajectory generated by the secondary system if the primary trajectory and the secondary trajectory are invalid. If no valid trajectory is determined, the vehicle may decelerate at a maximum rate.

Collision avoidance system with trajectory validation

Techniques for determining an error model based on vehicle data and ground truth data are discussed herein. To determine whether a complex system (which may be not capable of being inspected) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data. To provide safe operation of such a system, an error model can be determined that can provide a probability associated with perception data and a vehicle can determine a trajectory based on the probability of an error associated with the perception data.

Perception error models

Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.

Safety analysis framework

A vehicle computing system may implement techniques to predict behavior of objects detected by a vehicle operating in the environment. The techniques may include determining a feature with respect to a detected objects (e.g., likelihood that the detected object will impact operation of the vehicle) and/or a location of the vehicle and determining based on the feature a model to use to predict behavior (e.g., estimated states) of proximate objects (e.g., the detected object). The model may be configured to use one or more algorithms, classifiers, and/or computational resources to predict the behavior. Different models may be used to predict behavior of different objects and/or regions in the environment. Each model may receive sensor data as an input, and output predicted behavior for the detected object. Based on the predicted behavior of the object, a vehicle computing system may control operation of the vehicle.

Hyper planning based on object and/or region

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object or sensor limitation. An occluded region for an object is determined by the vehicle as part of an occlusion grid, from the perspective of the vehicle. The vehicle may receive another occlusion grid from another source, such as another vehicle or a remote computing device that stores and distributes occlusion grids. The other occlusion grid may be from a different perspective than the occlusion grid generated by the vehicle, and may include occupancy data for the region that is otherwise occluded from the perspective of the vehicle. The vehicle can be controlled to traverse the environment based on the occupancy data received from the other source.

Sharing occlusion data

Remote controlling of a vehicle, such as an autonomous vehicle, may sometimes be more efficient and/or reliable. Such control, however, may require processes for ensuring safety of surrounding persons and objects. Aspects of this disclosure include using onboard sensors to detect objects in an environment and alter remote commands according to such objects, e.g. by reducing a maximum permitted velocity of the vehicle as a function of distance to detected objects. In some examples described herein, such remote controlling may be performed by using objects in the environment as control objects, with movements of the control objects resulting in movement of the vehicle.

External control of an autonomous vehicle

Techniques for determining an error model associated with a system/subsystem of vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, errors can be introduced into operating regimes (scenarios) to validate the safe operation of such a system. By comparing captured and/or generated vehicle data with ground truth data, an error of the system can be statistically quantified and modeled. The statistical model can be used to introduce errors to the scenario to perturb the scenario to test, for example, a vehicle controller. Based on a simulation of the vehicle controlled in the perturbed scenario, a safety metric associated with the vehicle controller can be determined, as well as causes for any failures.

Growing community of inventors

San Jose, CA, United States of America

Marc Wimmershoff