Zoox, Inc.

Ravi Verma Gogna

Paul Orecchio

Meredith James Goldman

Jesse Sol Levinson

Ashutosh Gajanan Rege

Gary Linscott

Dan Xie

Amanda Lee Kelly Lockwood

Timothy Caldwell

Marin Kobilarov

Frank Reinaldo Ramirez

Matthew Miller Young

Rick Zhang

Dennis Schaefer

Arian Houshmand

Hristo Iankov Bojinov

Bryce Douglas Jones Carter

Jimmy Ming Chiang

Max Ellis Wolotsky

Subasingha Shaminda Subasingha

Sarah Tariq

Marc Wimmershoff

Caili Li

Niroth Keo

Joseph Funke

Collin MacGregor

Steven Cheng Qian

Joona Markus Petteri Kiiski

Mark Jonathon McClelland

Jacob Patrick Thalman

Udayan Pushparaj Karmarkar

Aaron Rosekind

Ruikun Yu

Matthew Nathaniel Dickerson

Satya Rama Aditya Pinapala

Collin Macgregor

Lakshay Garg

Wenhao Sheng

Muhammad Umar Choudry

Gerardo Cid Fernandez

Liang Chao

Eunsuk Chong

Oliver Fielding Douglas

Edward William O'Donnell

Techniques for detecting and responding to an emergency vehicle are discussed. A vehicle computing system may determine that an emergency vehicle based on sensor data, such as audio and visual data. In some examples, the vehicle computing system may determine aggregate actions of objects (e.g., other vehicles yielding) proximate the vehicle based on the sensor data. In such examples, a determination that the emergency vehicle is operating may be based on the actions of the objects. The vehicle computing system may, in turn, identify a location to move out of a path of the emergency vehicle (e.g., yield) and may control the vehicle to the location. The vehicle computing system may determine that the emergency vehicle is no longer relevant to the vehicle and may control the vehicle along a route to a destination. Determining to yield and/or returning to a mission may be confirmed by a remote operator.

Emergency vehicle detection and response

A vehicle computing system may identify an obstruction along a route of travel and may connect to a service computing device for guidance. The service computing device may include a guidance system configured to receive waypoint and/or orientation input from an operator. The operator may evaluate the scenario and determine one or more waypoints and/or associated orientations for the vehicle to navigate the scenario. In some examples, the guidance system may validate the waypoint(s) and/or associated orientation(s). The service computing device may send the waypoint(s) and/or associated orientation(s) to the vehicle computing system. The vehicle computing system may validate the waypoint(s) and/or associated orientation(s) and, based on the validation, control the vehicle according to the input. Based on a determination that the vehicle has navigated the scenario, the guidance system may release vehicle guidance back to the vehicle computing system.

Remote vehicle guidance

A computer implemented method is provided. The method, comprises: receiving from an autonomous vehicle sensor data, wherein the sensor data is indicative of an environment in which the vehicle is currently located or was previously located. The method further comprises receiving or determining additional data associated with at least one of: the sensor data, the vehicle, or the environment, wherein the additional data is different from the sensor data. The method further comprises displaying, on a display, an output comprising a representation of the sensor data. The method further comprises determining, based at least in part on the additional data, a reliability metric, the reliability metric being indicative of how reliable the sensor data is at representing the environment in which the vehicle is located at a current time. The method further comprises causing the output on the display to be based at least in part on the reliability metric.

Scene reliability

Techniques for intelligent vehicle pull over in response to detecting abnormal conditions. The techniques may include determining a presence of a condition necessitating that a vehicle cease from operating on a road surface and determining a constraint associated with the vehicle ceasing from operating on the road surface. In some examples, the constraint may be a time constraint, a geographical constraint, and/or the like. The techniques may also include sending, to a planner component of the vehicle, an indication that the vehicle is to cease from operating on the road surface in accordance with the constraint. Additionally, based at least in part on a determination that the planner component acknowledged the indication in accordance with the constraint, the techniques may include refraining from causing the vehicle to stop on the road surface such that the vehicle impedes a flow of traffic.

Intelligent vehicle pull over

Display screen or portion thereof having a graphical user interface

This disclosure is directed to techniques for collaborative driving during flagger interactions. For instance, a vehicle navigating along a path may detect a flagger using sensor data. Based on detecting the flagger, the vehicle may stop at a location that is at least a threshold distance from the flagger and also send the sensor data to one or more computing devices associated with a teleoperator. The vehicle may then receive a guidance from the one or more computing devices, where the guidance is to proceed past the location. In some examples, the vehicle receives the guidance when the teleoperator activates and holds an input device. Based on receiving the guidance, the vehicle may begin to navigate passed the location and by the flagger. In some examples, the vehicle continues to navigate as long as the teleoperator activates the input device.

Collaborative guidance for vehicle interactions

A vehicle can connect to multiple networks and multiple frequency bands and can determine network parameters (e.g., available bandwidth, throughput, latency, signal strength, etc.) associated with the multiple networks. Additionally, the vehicle can access network map data associated with the multiple networks. As the vehicle traverses an environment, the vehicle can collect sensor data of the environment and/or vehicle data (e.g., vehicle pose, diagnostic data, etc.). Based on the vehicle location and the network map data, the vehicle can optimize the use of networks and frequency bands to determine portions of the sensor data and/or vehicle data to transmit via the one or more of the multiple networks and multiple frequency bands.

Adaptive multi-network multi-band vehicle communications

Techniques and methods for securing vehicle systems. For instance, an authorization system may store data representing frequencies at which destination locations are associated pick-up locations for a fleet of autonomous vehicles. The authorization system may then receive a request for an autonomous vehicle to pick up a passenger at a first location and drop off the passenger at a second location. Based on the first location and the second location, the authorization system may determine a frequency for the request. The authorization system may then determine whether a control system for the fleet of autonomous vehicles is compromised based on whether the frequency is less than or equal to a threshold frequency. If the authorization system determines that the control system is compromised, the authorization system may perform a remedial action, such as notifying a teleoperator.

Techniques for authorizing vehicle control systems

Techniques for providing remote guidance to a vehicle operating in an environment, by an operator located in the environment, are described herein. The operator can include a safety observer configured to observe one or more vehicles operating in the environment and may identify a scenario that requires a modification to a vehicle operation (e.g., stop forward movement, change direction of travel, modify a maximum speed, etc.). The operator may access a graphical user interface (GUI) via an operator computing device, and may input a constraint to modify the vehicle operation. The vehicle computing system may receive a control signal including the constraint, and may modify a vehicle trajectory based on the constraint. The vehicle computing system may later determine that a condition associated with the constraint is satisfied, and may continue vehicular operation in absence of the constraint.

The present disclosure involves systems and methods for detecting road blockages and generating alternative routes. In some cases, a system detects, based at least in part on sensor data associated with an autonomous vehicle, a portion of an environment that impedes a planned path of the autonomous vehicle. The system determines a semantic classification associated with the portion of the environment and transmits a re-routing request comprising the semantic classification to one or more remote computing devices. The system receives an instruction associated with navigating the autonomous vehicle around the portion of the environment from the remote computing devices, where the instruction includes an alternative route determined from a plurality of alternative routes. The system further controls the autonomous vehicle to navigate around the portion of the environment based at least in part on the instruction.

Techniques for detecting road blockages and generating alternative routes

Techniques to provide guidance to a vehicle operating in an environment are discussed herein. For example, such techniques may include sending a request for assistance, receiving a reference trajectory, and causing the vehicle to determine a trajectory based on the reference trajectory. Data such as sensor data and vehicle state data may be sent from the vehicle to a remote computing device. The computing device outputs a user interface using the data and determines the reference trajectory based on receiving an input in the user interface. The techniques can send an indication of the reference trajectory to the vehicle for use in planning a trajectory for the vehicle. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory.

Collaborative vehicle guidance

Techniques for providing remote assistance to a vehicle are discussed. The techniques include receiving, from a vehicle, an indication of an event and displaying, on a display and to a user, a portion of an environment including the vehicle. The techniques further determine a valid region in the portion of the environment associated with a location at which the vehicle is capable of navigating. The techniques also display, on the display a footprint of the vehicle, where the footprint is associated with a position and orientation. The techniques further include transmitting the position and orientation of the footprint to the vehicle, which causes the vehicle to traverse in the environment to the position and orientation.

Remote assistance for vehicles

Techniques for validating and correcting teleoperation signals for an autonomous vehicle are described herein. In some examples, a system may receive, from a remote computing device, an instruction to stop movement of an autonomous. The system may determine, based at least in part on map data, that the instruction is associated with stopping the autonomous vehicle in a non-stopping area. In response to determining that the instruction is associated with stopping the autonomous vehicle in the non-stopping area, the system may determine to continue movement of the autonomous vehicle beyond the no-stopping area. The system may further identify a stopping location that is at least partially outside of the no-stopping area and control the autonomous vehicle to instead stop at the stopping location.

Validating stopping behaviors of a vehicle

A teleoperations system that collaboratively works with an autonomous vehicle planning component to generate a path for controlling the autonomous vehicle to pass a situation where the vehicle is unable to identify a vehicle option to proceed and may comprise presenting one or more paths to a teleoperator (e.g., a human user, machine-learned model, and/or artificial intelligence component), such paths being generated either at the vehicle or remote system. The teleoperations system may receive input from the teleoperator indicating a vehicle option to select for the vehicle to proceed in the environment. The teleoperations system may generate a guidance path based on the vehicle options and the input and transmit the guidance path to the autonomous vehicle. Based at least in part on the guidance path, the autonomous vehicle may generate a control trajectory to use to navigate around the obstacle.

Collaborative action ambiguity resolution for autonomous vehicles

A vehicle can connect to multiple networks and can determine network parameters (e.g., available bandwidth, latency, signal strength, etc.) associated with the multiple networks. Additionally, the vehicle can access network map data associated with the multiple networks. As the vehicle traverses an environment, the vehicle can collect sensor data of the environment and/or vehicle data (e.g., vehicle pose, diagnostic data, etc.). Based on the network parameters and the network map data, the vehicle can optimize the use of the networks determine portions of the sensor data and/or vehicle data to transmit via the one or more of the multiple networks.

Adaptive multi-network vehicle architecture

Growing community of inventors

San Jose, CA, United States of America

Ravi Verma Gogna