Zoox, Inc.

Paul Orecchio

Ravi Verma Gogna

Jesse Sol Levinson

Ashutosh Gajanan Rege

Gary Linscott

Dan Xie

Amanda Lee Kelly Lockwood

Marin Kobilarov

Timothy Caldwell

Meredith James Goldman

Arian Houshmand

Muhammad Umar Choudry

Gerardo Cid Fernandez

Liang Chao

A computer implemented method is provided. The method, comprises: receiving from an autonomous vehicle sensor data, wherein the sensor data is indicative of an environment in which the vehicle is currently located or was previously located. The method further comprises receiving or determining additional data associated with at least one of: the sensor data, the vehicle, or the environment, wherein the additional data is different from the sensor data. The method further comprises displaying, on a display, an output comprising a representation of the sensor data. The method further comprises determining, based at least in part on the additional data, a reliability metric, the reliability metric being indicative of how reliable the sensor data is at representing the environment in which the vehicle is located at a current time. The method further comprises causing the output on the display to be based at least in part on the reliability metric.

Scene reliability

A vehicle can connect to multiple networks and multiple frequency bands and can determine network parameters (e.g., available bandwidth, throughput, latency, signal strength, etc.) associated with the multiple networks. Additionally, the vehicle can access network map data associated with the multiple networks. As the vehicle traverses an environment, the vehicle can collect sensor data of the environment and/or vehicle data (e.g., vehicle pose, diagnostic data, etc.). Based on the vehicle location and the network map data, the vehicle can optimize the use of networks and frequency bands to determine portions of the sensor data and/or vehicle data to transmit via the one or more of the multiple networks and multiple frequency bands.

Adaptive multi-network multi-band vehicle communications

A teleoperations system that collaboratively works with an autonomous vehicle planning component to generate a path for controlling the autonomous vehicle to pass a situation where the vehicle is unable to identify a vehicle option to proceed and may comprise presenting one or more paths to a teleoperator (e.g., a human user, machine-learned model, and/or artificial intelligence component), such paths being generated either at the vehicle or remote system. The teleoperations system may receive input from the teleoperator indicating a vehicle option to select for the vehicle to proceed in the environment. The teleoperations system may generate a guidance path based on the vehicle options and the input and transmit the guidance path to the autonomous vehicle. Based at least in part on the guidance path, the autonomous vehicle may generate a control trajectory to use to navigate around the obstacle.

Collaborative action ambiguity resolution for autonomous vehicles

A vehicle can connect to multiple networks and can determine network parameters (e.g., available bandwidth, latency, signal strength, etc.) associated with the multiple networks. Additionally, the vehicle can access network map data associated with the multiple networks. As the vehicle traverses an environment, the vehicle can collect sensor data of the environment and/or vehicle data (e.g., vehicle pose, diagnostic data, etc.). Based on the network parameters and the network map data, the vehicle can optimize the use of the networks determine portions of the sensor data and/or vehicle data to transmit via the one or more of the multiple networks.

Adaptive multi-network vehicle architecture

A method for autonomously operating a driverless vehicle along a path between a first geographic location and a destination may include receiving communication signals from the driverless vehicle. The communication signals may include sensor data from the driverless vehicle and data indicating occurrence of an event associated with the path. The communication signals may also include data indicating that a confidence level associated with the path is less than a threshold confidence level due to the event. The method may also include determining, via a teleoperations system, a level of guidance to provide the driverless vehicle based on data associated with the communication signals, and transmitting teleoperations signals to the driverless vehicle. The teleoperations signals may include guidance to operate the driverless vehicle according to the determined level of guidance, so that a vehicle controller maneuvers the driverless vehicle to avoid, travel around, or pass through the event.

Interactions between vehicle and teleoperations system

A driverless vehicle may include a processor, a sensor, a network interface, and a memory having stored thereon processor-executable instructions. The driverless vehicle may be configured to obtain a stream of sensor signals including sensor data related to operation of the driverless vehicle from the sensor and/or the network interface. The driverless vehicle may be configured to determine a confidence level associated with operation of the driverless vehicle from the sensor data, and store the confidence level and at least a portion of the sensor data. The driverless vehicle may also be configured to transmit via the network interface a request for teleoperator assistance, and the request may include the portion of the sensor data and the confidence level.

Teleoperator situational awareness

Techniques and methods for navigating an autonomous vehicle using teleoperator instructions. For instance, while navigating to a location, progress of an autonomous vehicle may stop. This may be caused by the autonomous vehicle yielding to another vehicle, such as at an intersection or when the other vehicle is located along a route of the autonomous vehicle. After yielding for a threshold amount of time, the autonomous vehicle may send sensor data and/or an indication of the other vehicle to the teleoperator. The autonomous vehicle may then receive, from the teleoperator, an instruction to cease yielding to the other vehicle. Based at least in part on the instruction, the autonomous vehicle may again begin navigating.

Techniques for navigating vehicles using teleoperator instructions

Techniques and methods for contacting a teleoperator. For instance, while navigating to a location, progress of an autonomous vehicle may stop. This may be caused by the autonomous vehicle yielding to another vehicle, such as at an intersection or when the other vehicle is located along a route of the autonomous vehicle. While yielding to the other vehicle, the autonomous vehicle may determine that the progress has stopped for a threshold amount of time. Additionally, in some circumstances, the autonomous vehicle may determine that the progress not been stopped due to traffic or a traffic light. Based at least in part on the determinations, the autonomous vehicle may send sensor data to the teleoperator, where the sensor data represents an environment for which the autonomous vehicle is located. Additionally, the autonomous vehicle may send an indication of the other vehicle to the teleoperator.

Techniques for contacting a teleoperator

A teleoperator device may be configured to obtain a request for teleoperator assistance from a driverless vehicle and obtain teleoperator data in response to the request. The teleoperator device may also be configured to record at least some of the teleoperator input and/or guidance transmitted to the driverless vehicle based on the teleoperator input. Upon receiving a subsequent request, the teleoperator device may be configured to reproduce at least part of the former teleoperator input and/or to provide an option to activate guidance associated with the teleoperator input. The teleoperator device may also be configured to train a model and/or use a model to determine from vehicle data an option for presentation via a teleoperator interface and/or a presentation configuration of the teleoperator interface.

Predictive teleoperator situational awareness

A method for operating a driverless vehicle may include receiving, at the driverless vehicle, sensor signals related to operation of the driverless vehicle, and road network data from a road network data store. The method may also include determining a driving corridor within which the driverless vehicle travels according to a trajectory, and causing the driverless vehicle to traverse a road network autonomously according to a path from a first geographic location to a second geographic location. The method may also include determining that an event associated with the path has occurred, and sending communication signals to a teleoperations system including a request for guidance and one or more of sensor data and the road network data. The method may include receiving, at the driverless vehicle, teleoperations signals from the teleoperations system, such that the vehicle controller determines a revised trajectory based at least in part on the teleoperations signals.

Growing community of inventors

Cupertino, CA, United States of America

Paul Orecchio