Zoox, Inc.

Mahsa Ghafarianzadeh

Benjamin John Sapp

James William Vaisey Philbin

Andrew Scott Crego

Abishek Krishna Akella

Kenneth Michael Siebert

Luke Martin Hansen

Kai Zhenyu Wang

Siavosh Rezvan Behbahani

Gowtham Garimella

Gerrit Bagschik

Rasmus Fonseca

Muhammad Farooq Rama

Techniques for determining that a first vehicle is associated with a reverse state, and controlling a second vehicle based on the reverse state, are described herein. In some examples, the first vehicle may provide an indication that the first vehicle will be executing a reverse maneuver, such as with reverse lights on the vehicle or by positioning at an angle relative to a road or parking space to allow for the reverse maneuver into a desired location. A planning system of the second vehicle (such as an autonomous vehicle) may receive sensor data and determine a variety of these indications to determine a probability that the vehicle is going to execute a reverse maneuver. The second vehicle can further determine a likely trajectory of the reverse maneuver and can provide appropriate accommodations (e.g., time and/or space) to allow the second vehicle to execute the maneuver safely and efficiently.

Planning accommodations for reversing vehicles

A method and system of determining whether a stationary vehicle is a blocking vehicle to improve control of an autonomous vehicle. A perception engine may detect a stationary vehicle in an environment of the autonomous vehicle from sensor data received by the autonomous vehicle. Responsive to this detection, the perception engine may determine feature values of the environment of the vehicle from sensor data (e.g., features of the stationary vehicle, other object(s), the environment itself). The autonomous vehicle may input these feature values into a machine-learning model to determine a probability that the stationary vehicle is a blocking vehicle and use the probability to generate a trajectory to control motion of the autonomous vehicle.

Detecting vehicle aperture and/or door state

Techniques for modeling the probability distribution of errors in perception systems are discussed herein. For example, techniques may include modeling error distribution for attributes such as position, size, pose, and velocity of objects detected in an environment, and training a mixture model to output specific error probability distributions based on input features such as object classification, distance to the object, and occlusion. The output of the trained model may be used to control the operation of a vehicle in an environment, generate simulations, perform collision probability analyses, and to mine log data to detect collision risks.

Perception error modeling

Detecting blocking objects

Techniques are discussed for predicting locations of an object based on attributes of the object and/or attributes of other object(s) proximate to the object. The techniques can predict locations of a pedestrian proximate to a crosswalk as they traverse or prepare to traverse through the crosswalk. The techniques can predict locations of objects as the object traverses an environment. Attributes can comprise information about an object, such as a position, velocity, acceleration, classification, heading, relative distances to regions or other objects, bounding box, etc. Attributes can be determined for an object over time such that, when a series of attributes are input into a prediction component (e.g., a machine learned model), the prediction component can output, for example, predicted locations of the object at times in the future. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the predicted locations.

Prediction based on attributes

Simulating realistic movement of an object, such as a vehicle or pedestrian, that accounts for unusual behavior may comprise generating an agent behavior model based at least in part on output of a perception component of an autonomous vehicle and determining a difference between the output and log data that includes indications of an actual maneuver of location of an object. Simulating movement of an object may comprise determining predicted motion of the object using the perception component and modifying the predicted motion based at least in part on the agent behavior model.

Agent behavior model for simulation control

Pedestrian prediction based on attributes

Systems and methods for segmenting an image using a convolutional neural network are described herein. A convolutional neural network (CNN) comprises an encoder-decoder architecture, and may comprise one or more Long Short Term Memory (LSTM) layers between the encoder and decoder layers. The LSTM layers provide temporal information in addition to the spatial information of the encoder-decoder layers. A subset of a sequence of images is input into the encoder layer of the CNN and a corresponding sequence of segmented images is output from the decoder layer. In some embodiments, the one or more LSTM layers may be combined in such a way that the CNN is predictive, providing predicted output of segmented images. Though the CNN provides multiple outputs, the CNN may be trained from single images or by generation of noisy ground truth datasets. Segmenting may be performed for object segmentation or free space segmentation.

Growing community of inventors

Emerald Hills, CA, United States of America

Mahsa Ghafarianzadeh