Zoox, Inc.

Benjamin John Sapp

James William Vaisey Philbin

Mahsa Ghafarianzadeh

Xi Joey Hong

Kai Zhenyu Wang

Yilun Wang

Daylen Guang Yu Yang

A method and system of determining whether a stationary vehicle is a blocking vehicle to improve control of an autonomous vehicle. A perception engine may detect a stationary vehicle in an environment of the autonomous vehicle from sensor data received by the autonomous vehicle. Responsive to this detection, the perception engine may determine feature values of the environment of the vehicle from sensor data (e.g., features of the stationary vehicle, other object(s), the environment itself). The autonomous vehicle may input these feature values into a machine-learning model to determine a probability that the stationary vehicle is a blocking vehicle and use the probability to generate a trajectory to control motion of the autonomous vehicle.

Detecting vehicle aperture and/or door state

Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.

Trajectory prediction on top-down scenes and associated model

Detecting blocking objects

Techniques are discussed for determining predicted trajectories based on a top-down representation of an environment. Sensors of a first vehicle can capture sensor data of an environment, which may include agent(s) separate from the first vehicle, such as a second vehicle or a pedestrian. A multi-channel image representing a top-down view of the agent(s) and the environment and comprising semantic information can be generated based on the sensor data. Semantic information may include a bounding box and velocity information associated with the agent, map data, and other semantic information. Multiple images can be generated representing the environment over time. The image(s) can be input into a prediction system configured to output a heat map comprising prediction probabilities associated with possible locations of the agent in the future. A predicted trajectory can be generated based on the prediction probabilities and output to control an operation of the first vehicle.

Trajectory prediction on top-down scenes

The generation and/or use of probabilistic heat maps for use in predicting the behavior of entities in an environment is described. In an example, a computing device(s) can receive sensor data from sensors of a vehicle in an environment. The computing device(s) can determine, based at least in partly on the sensor data, a location of an entity in the environment, and a first characteristic associated with the entity or the environment (type, velocity, orientation, etc.). The computing device(s) can access, from a database, a heat map generated from previously collected sensor data associated with the environment. The computing device(s) can perform a look-up using the heat map based at least partly on the location of the entity and the first characteristic, and can determine a predicted behavior of the entity at a predetermined future time based on a pattern of behavior associated with a cell in the heat map.

Probabilistic heat maps for behavior prediction

Feature-based prediction is described. In an example, a vehicle can capture sensor data while traversing an environment and can provide the sensor data to computing system(s). The sensor data can indicate event(s), such as a lane change, associated with agent(s) in the environment. The computing system(s) can determine, based on the sensor data, a time associated with the event and can determine features associated with a period of time relative to the time of the event. In an example, the computing system(s) can aggregate the features with additional features associated with other similar events to generate training data and can train, based at least in part on the training data, a machine learned model for predicting new events. In an example, the machine learned model can be transmitted to vehicle(s), which can be configured to alter drive operation(s) based, at least partly, on output(s) of the machine learned model.

Feature-based prediction

A temporal prediction model for semantic intent understanding is described. An agent (e.g., a moving object) in an environment can be detected in sensor data collected from sensors on a vehicle. Computing device(s) associated with the vehicle can determine, based partly on the sensor data, attribute(s) of the agent (e.g., classification, position, velocity, etc.), and can generate, based partly on the attribute(s) and a temporal prediction model, semantic intent(s) of the agent (e.g., crossing a road, staying straight, etc.), which can correspond to candidate trajectory(s) of the agent. The candidate trajectory(s) can be associated with weight(s) representing likelihood(s) that the agent will perform respective intent(s). The computing device(s) can use one (or more) of the candidate trajectory(s) to determine a vehicle trajectory along which a vehicle is to drive.

Growing community of inventors

San Francisco, CA, United States of America

Benjamin John Sapp